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EXECUTIVE  SUMMARY 

This  report  describes  in  detail  the  procedures,  methods,  and  resources  Parsons  used  to  complete 
an  Environmental  Security  Technology  Certification  Program  (ESTCP)-funded  demonstration 
study  at  the  former  Waikoloa  Maneuver  Area  (WMA)  in  Waikoloa,  Hawaii,  under  ESTCP 
Munitions  Response  Project  201104.  The  2013  demonstration  study  at  the  former  WMA  was 
conducted  with  two  primary  objectives: 

•  Test  and  validate  detection  and  discrimination  capabilities  of  a  currently  available 
advanced  electromagnetic  induction  sensor  developed  specifically  for  discrimination  on 
real  sites  under  operational  conditions. 

•  Investigate  in  cooperation  with  regulators  and  program  managers  how  classification 
technologies  can  be  implemented  in  munitions  and  explosives  of  concern  (MEC)  cleanup 
operations. 

Parsons  was  responsible  for  all  of  the  field  work  conducted  on  the  project,  which  included 
EM61-MK2  (EM61)  transect  surveys  of  prospective  MetalMapper  data  collection  locations,  the 
placement  of  97  seed  items  for  use  in  measuring  the  capabilities  of  the  MetalMapper  at  the  site, 
the  collection  of  dynamic  EM61  data  and  selection  of  targets  in  the  data,  the  cued  survey  of 
1,032  dynamic  targets  with  the  MetalMapper,  and  the  intrusive  investigation  of  the  target  sources 
following  data  collection.  While  the  EM61  transect  surveys  were  conducted  as  planned, 
MetalMapper  data  collection  locations  were  based  almost  solely  on  accessibility  to  the 
MetalMapper.  The  former  WMA  is  primarily  composed  of  jagged  volcanic  rock,  and  most  of  it 
could  not  be  safely  traversed  by  the  extendable  reach  forklift  used  to  transport  the  sensor.  Three 
separate  locations  totaling  approximately  5  acres  of  usable  land  were  identified  for  survey  within 
the  site.  Even  within  these  relatively  flat  and  even  areas,  the  rocky  surface  was  deemed 
unsuitable  for  the  collection  of  dynamic  MetalMapper  data  as  originally  planned,  so  the  dynamic 
data  were  collected  using  the  EM61.  Cued  MetalMapper  data  were  collected  as  specified  in  the 
Demonstration  Plan. 

The  EM61  and  MetalMapper  data  collection  effort  took  place  over  6  weeks  and  was  completed 
by  two  field  teams.  One  team  began  the  dynamic  EM61  data  collection  and  performed  all  of  the 
cued  MetalMapper  data  collection;  the  second  mobilized  during  the  project  and  completed  the 
dynamic  data  collection  while  the  first  began  the  MetalMapper  work.  Production  rates  were 
extremely  slow  for  both  dynamic  and  cued  data  collection.  Dynamic  data  collection  was  slowed 
by  the  site  terrain  and  the  narrow  line  spacing  required  to  identify  and  accurately  position  smaller 
ordnance  types  like  37-millimeter  projectiles.  The  cued  data  collection  was  slowed  by  the  terrain 
and  the  site  geology,  which  made  identifying  subsurface  targets  difficult  when  they  were  present 
and  resulted  in  lengthy  searches  for  dynamic  targets  that  were  selected  based  on  geologic 
response  rather  than  subsurface  metal. 

The  cued  MetalMapper  data  were  analyzed  by  Black  Tusk  Geophysics  (BTG),  which  provided 
an  initial  dig  list  for  the  site.  The  dig  list  was  not  fiilly  classified  but  did  include  BTG’s  initial 
impressions  on  whether  each  target  appeared  to  be  caused  by  an  actual  metallic  source  or 
appeared  to  be  an  inversion  of  background.  The  intrusive  investigation  was  performed  following 
the  submittal  of  this  dig  list.  Targets  classified  as  metallic  sources  by  BTG  were  investigated 
using  typical  ESTCP  intrusive  investigation  protocols.  Those  classified  as  likely  background 
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were  checked  with  a  Minelab  metal  detector  before  intrusive  investigation.  If  the  Minelab  also 
indicated  that  there  was  no  metallic  source  present,  no  intrusive  investigation  was  performed. 

The  dig  list  submitted  by  BTG  was  scored  against  the  ground  truth  from  the  intrusive 
investigation  and  the  blind  seeds  present  in  the  survey  area.  Nine  seed  items  were  incorrectly 
classified  as  nonhazardous  clutter  on  BTG’s  dig  list.  Failure  analysis  indicated  that  the  incorrect 
classification  on  one  of  the  nine  targets  was  due  to  significant  offset  between  the  MetalMapper 
data  collection  point  and  the  actual  location  of  the  seed.  The  other  eight  incorrect  classifications 
appeared  to  be  a  result  of  the  variable  background  response  produced  by  the  iron-rich  volcanic 
rock  at  the  site.  Two  were  identified  as  metal  that  did  not  match  any  library  objects  particularly 
well  and  were  classified  as  clutter;  the  remaining  six  appeared  to  be  purely  derived  from  soil 
response  and  were  classified  as  background.  The  initial  approach  taken  by  BTG  in  analyzing  the 
data  included  various  methods  of  removing  background,  so  there  are  no  obvious  methods  for 
reprocessing  the  data  to  resolve  just  the  response  due  to  the  seed  items  for  these  targets.  BTG’s 
conclusion  was  that  the  extreme  variations  in  background  across  the  site  affected  the  data  to  the 
degree  that  these  effects  could  not  be  overcome.  It  was  recommended  that  all  anomalies 
identified  in  the  EM61  detection  survey  be  investigated  to  ensure  that  all  TOI  were  recovered. 
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1.0  INTRODUCTION 


Currently,  up  to  90%  of  excavation  costs  on  most  unexploded  ordnance  (UXO)  /  munitions  and 
explosives  of  concern  (MEC)  projects  are  related  to  removing  scrap  metal  that  does  not  represent 
an  explosive  hazard.  Significant  cost  savings  could  be  achieved  through  the  use  of  geophysical 
discrimination  methods  that  could  reduce  the  number  of  excavations  required  to  remove 
explosive  hazards  from  sites.  The  objective  of  this  project  is  to  demonstrate  the  use  of  advanced 
electromagnetic  induction  (EMI)  sensors  in  dynamic  and  static  data  acquisition  modes  and 
associated  analysis  software.  To  achieve  these  objectives,  a  controlled  test  was  conducted  at  the 
former  Waikoloa  Maneuver  Area  (WMA)  in  Waikoloa,  Hawaii. 

This  project  was  performed  as  a  demonstration  study  funded  by  the  Environmental  Security 
Technology  Certification  Program  (ESTCP)  under  Munitions  Response  Project  201104.  This 
demonstration  was  designed  to  evaluate  classification  methods  at  a  site  that  is  known  to  contain 
various  munitions,  including  evidence  of  ordnance  as  small  as  37-millimeter  (mm)  projectiles, 
presents  extremely  variable  terrain  and  geology,  and  provides  an  opportunity  to  involve  a 
stakeholder  community  that  includes  state  regulators  in  the  classification  demonstration  program. 

1.1  BACKGROUND 

The  Fiscal  Year  2006  defense  appropriation  contained  funding  for  the  “Development  of 
Advanced,  Sophisticated  Discrimination  Technologies  for  UXO  Cleanup.”  The  ESTCP 
responded  by  conducting  a  UXO  discrimination  study  at  the  former  Camp  Sibert,  Alabama.  The 
results  of  this  first  demonstration  were  very  encouraging.  The  conditions  for  discrimination  were 
favorable  at  this  site  and  included  a  single  target  of  interest  (TOI),  the  4.2-inch  (in.)  mortar,  and 
benign  topography  and  geology.  All  of  the  classification  approaches  demonstrated  were  able  to 
correctly  identify  a  sizable  fraction  of  the  anomalies  as  arising  from  nonhazardous  items  that 
could  be  safely  left  in  the  ground.  Both  commercial  and  advanced  sensors  produced  very  good 
results.  ESTCP  organized  demonstrations  at  munitions  response  sites  across  the  country  between 
2006  and  2013,  generally  with  new  variables  added  to  the  classification  challenges  at  each 
subsequent  site  (i.e.,  increased  target  density,  increased  response  from  local  geology,  mixed 
munition  sizes  ranging  from  small  to  very  large,  wooded  areas).  In  addition,  the  subsequent 
projects  included  the  use  of  smaller,  man-portable  EMI  sensors  such  as  the  Naval  Research 
Laboratory’s  TEMTADS  2x2  cart,  Lawrence  Berkeley  National  Laboratory’s  man-portable 
Berkeley  UXO  discriminator  (BUD),  and  Sky  Research’s  man-portable  vector  machine.  All  of 
the  EMI  sensors  tested  to  date  have  been  quite  successful  in  discriminating  between  TOI  and 
clutter. 

The  earlier  demonstration  projects  focused  on  proving  that  the  technology  was  effective  by 
comparing  theoretical  dig  lists  to  real-world  sources  by  excavating  all  of  the  targets  at  a  given 
site  and  comparing  the  known  source  results  to  the  predicted  source  results.  More  recent  projects 
have  focused  on  actually  leaving  metal  classified  as  non-TOI  in  the  ground  following  the 
completion  of  the  project.  An  ESTCP/  United  States  Army  Corps  of  Engineers  (USACE)-funded 
Pilot  Study  at  the  former  Camp  Beale  in  California  and  a  non-ESTCP-related  resulted  removal 
action  performed  at  two  sites  at  the  former  Camp  Sibert  resulted  in  more  than  7,000  dynamic 
target  sources  remaining  un-dug  at  both  sites,  with  regulator  approval.  No  TOI  were 
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misclassified  at  either  site,  and  before  the  addition  of  quality  assuranee  verification  digs,  the 
reduction  in  necessary  clutter  digs  was  above  90%  for  each. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

This  type  of  approach  has  the  potential  to  reduce  the  number  of  excavations  required  to 
effectively  remove  the  explosive  safety  risk  (MEC)  at  a  given  site,  which  would  result  in 
significant  cost  savings  related  to  the  closure  of  formerly  used  defense  sites  (FUDS).  The  cost 
savings  are  expected  to  be  particularly  significant  at  removal  action  sites  such  as  the  WMA, 
where  the  current  cost  of  cleanup  is  expected  to  approach  $1  billion  using  current  removal 
methods. 

1.3  REGULATORY  DRIVERS 

As  part  of  the  cleanup  of  former  Department  of  Defense  sites,  buy-in  is  required  from  regulatory 
agencies  at  the  federal,  state,  and  local  levels.  The  advancement  in  classification  sensors  and 
their  successful  deployment  at  real-world  sites  needs  to  be  documented  for  their  use  to  be 
accepted  by  the  applicable  regulatory  agencies.  Their  acceptance  of  the  use  of  this  technology  at 
sites  for  which  they  are  ultimately  responsible  will  be  particularly  important  with  the  potential 
for  Department  of  Defense  budget  cuts  to  affect  the  amount  of  money  that  will  be  available  for 
future  remedial  actions. 


2 


2.0  TECHNOLOGY 


2.1  TECHNOLOGY  DESCRIPTION 

The  MetalMapper  is  an  advanced  EMI  system  developed  by  Geometries,  Inc.,  with  support  from 
the  ESTCP.  The  MetalMapper  draws  elements  of  its  design  from  advanced  systems  currently 
being  developed  by  G&G  Sciences,  Inc.  (supported  by  Naval  Sea  Systems  Command,  the 
Strategic  Environmental  Research  and  Development  Program,  and  ESTCP)  and  by  Lawrence 
Berkeley  National  Laboratory  with  support  from  the  Strategic  Environmental  Research  and 
Development  Program  and  ESTCP.  It  has  three  mutually  orthogonal  transmit  loops  in  the  Z,  Y, 
and  X  directions  and  contains  seven  triaxial  receiver  antennas  inside  the  Z  (bottom)  loop. 
Typically,  the  transmit  loops  are  driven  with  a  classical  bipolar  pulse-type  time  domain 
electromagnetic  waveform  (i.e.,  alternating  pulse  polarity  with  a  50%  duty  cycle).  Depending  on 
the  survey  mode  (e.g.,  static/dynamic),  the  fundamental  frequency  of  transmission  can  be  varied 
over  the  range  1.11  <f  <810  hertz  (Hz).  The  seven  receiver  antennas  allow  21  independent 
measurements  of  the  transient  secondary  magnetic  field. 

The  data  acquisition  computer  (DAQ)  is  built  around  a  commercially  available  product  from 
National  Instruments.  The  National  Instruments  DAQ  is  a  full-featured  PC  running  Windows  7. 
The  DAQ,  electromagnetic  transmitter,  and  batteries  for  the  system  are  packaged  in  an  aluminum 
case  that  can  be  mounted  on  a  pack  frame,  on  a  separate  cart  such  as  a  hand  truck,  or  on  the 
survey  vehicle  such  a  tractor.  The  instrumentation  package  also  includes  two  external  modules 
that  provide  real-time  kinematic  global  positioning  system  location  and  platform  attitude  (i.e., 
magnetic  heading,  pitch,  and  roll)  data.  These  modules  are  connected  to  the  DAQ  through  serial 
RS232C  ports.  A  block  diagram  of  the  DAQ  system  is  in  Figure  2-1. 

The  MetalMapper  has  two  modes  of  data  collection:  dynamic  and  static.  Data  collected  in 
dynamic  mode  results  in  data  files  containing  many  data  samples.  Generally  speaking,  dynamic 
mode  data  are  collected  while  the  antenna  platform  is  in  motion.  Static  mode  data  collection  is 
employed  for  cued  surveys.  As  its  name  implies,  the  antenna  platform  remains  static  or 
motionless  during  the  period  of  data  acquisition.  Depending  on  the  acquisition  parameters  (e.g., 
sample  period  and  stacking  parameter)  it  can  take  tens  of  seconds  to  complete  a  static 
measurement.  The  results  of  the  static  measurement  are  written  into  a  binary  data  file  containing 
only  a  single  data  point  representing  the  average  (stacked)  result,  usually  over  tens  or  even 
hundreds  of  repetitions  of  the  transmitter’s  base  frequency. 
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Figure  2-1:  DAQ  and  DAQ  Functional  Block  Diagram 


Data  are  acquired  in  time  blocks  that  consist  of  a  fixed  number  of  transmitter  cycle  “repeats.” 
Both  the  period  (T)  and  the  repeat  factor  (N)  are  operator  selectable  and  are  varied  in 
multiplicative  factors  of  3.  The  MetalMapper  also  averages  an  operator-specified  number  of 
acquisition  blocks  (NStacks)  together  before  the  acquired  data  are  saved  to  disk.  The  decay 
transients  that  are  received  during  the  off  times  are  stacked  (averaged)  with  appropriate  sign 
changes  for  positive  and  negative  half  cycles.  The  decays  in  an  individual  acquisition  block  are 
stacked,  and  the  decays  in  that  block  are  averaged  with  other  acquisition  blocks  (assuming  the 
operator  has  selected  NStack  greater  than  one).  The  resultant  data  are  saved  as  a  data  point.  A 
photo  of  the  typical  configuration  of  the  instrument  used  for  collecting  both  dynamic  and  cued 
data  is  shown  in  Figure  2-2. 


Figure  2-2:  Antenna  Array  and  Deployment  of  the  MetalMapper  at  WMA 
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In  its  present  (third  generation)  form,  the  MetalMapper  has  been  demonstrated  and  scored  at 
numerous  live  site  demonstrations  carried  out  by  ESTCP.  The  performance  of  the  MetalMapper 
at  these  sites  is  documented  in  formal  reports  issued  by  the  various  contractors  working  on  those 
projects. 

2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

There  are  a  few  advanced  EMI  sensors  that  are  similar  to  the  MetalMapper  in  theory,  design  and 
size,  with  the  most  comparable  being  the  TEMTADS  5x5  and  the  full-size  BUD.  The 
TEMTADS  5x5  consists  of  25  pairs  of  transmit/receive  coils  oriented  in  a  5x5  grid  pattern, 
approximately  2  meters  (m)  to  a  side.  The  BUD  is  composed  of  three  orthogonal  transmitters  and 
eight  pairs  of  differenced  receivers.  These  instruments  have  been  part  of  the  ongoing  ESTCP 
classification  demonstrations,  and  similar  results  have  been  documented  for  all  three  during 
previous  projects.  The  main  advantage  of  the  MetalMapper  is  that  it  is  currently  commercially 
available,  while  the  other  two  advanced  EMI  sensors  are  generally  only  used  by  the 
organizations  that  developed  them. 

The  greatest  limitation  of  the  MetalMapper  is  its  size,  both  of  the  sensor  itself  and  of  the 
accompanying  computer,  screen,  and  cables.  The  system  is  designed  primarily  for  use  in 
relatively  flat,  open  areas  and  cannot  currently  be  used  effectively  in  wooded  areas,  steep  terrain, 
or  extremely  rocky  terrain  where  the  transport  vehicles  used  for  the  sensor  cannot  maneuver 
effectively.  The  extendable-reach  forklift  used  to  transport  the  system  at  the  WMA  (Figure  2.2) 
was  effective  in  the  areas  selected  for  the  demonstration,  but  significant  effort  was  spent 
identifying  even  5.5  acres  of  relatively  flat,  somewhat  non-rocky  ground  over  which  the  system 
could  be  used  at  this  site.  Even  after  painstaking  site  selection,  it  was  determined  that  the  base  of 
the  bucket  in  which  the  MetalMapper  sits  during  data  collection  would  wear  down  almost  daily 
if  dynamic  data  collection  was  performed  with  the  MetalMapper,  so  the  decision  was  made  to 
use  the  EM61  for  the  dynamic  survey. 

In  addition  to  transport  issues,  the  sensor  can  be  extremely  difficult  to  repair  in  the  field.  In  the 
case  of  the  WMA  project,  one  of  the  receivers  failed  completely  (i.e.,  all  three  of  the  tri-axial 
wrappings)  at  the  beginning  of  the  project.  The  problem  was  diagnosed  in  the  field  but  could  not 
be  fixed  without  sending  the  MetalMapper  computer  back  to  Geometries  for  repair.  The  broken 
receiver  (Rx  0)  was  one  of  the  outer  receivers  in  the  unit,  and  after  consultation  with  the  USAGE 
geophysicist  on-site  and  Black  Tusk  Geophysics  (BTG),  it  was  determined  that  data  collection 
could  continue  without  repair  with  the  understanding  that  data  from  the  broken  receiver  would 
not  be  available  for  use  in  processing.  Various  wrappings  of  the  tri-axial  receivers  have  failed  on 
other  Parsons  projects  as  well,  although  all  failures  have  been  for  wrappings  on  outer  receivers, 
and  the  data  were  not  significantly  affected  as  long  as  the  data  from  the  malfunctioning 
wrapping(s)  were  not  used  in  inversion.  It  is  assumed  this  would  have  been  a  much  larger 
problem  had  the  issue  been  with  one  of  the  three  middle  receivers.  Data  collection  could  be  put 
on  hold  for  a  week  or  more  if  the  sensor/computer  needs  to  be  shipped  to  Geometries  from  a  site 
like  WMA,  and  it  is  nearly  impossible  to  perform  repairs  to  either  the  sensor  or  the  computer  in 
the  field. 
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3.0  PERFORMANCE  OBJECTIVES 

The  specific  performance  objectives  for  this  project  are  summarized  in  Table  3-1. 


Table  3-1:  Performance  Objectives  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Dynamic  Data  Collection  Objectives 

Repeatability  of 
instrument 
verification  strip 
(IVS) 

measurements 

Amplitude  of 
electromagnetic  anomaly 

Twice-daily  IVS  survey 
data 

Detection:  Amplitude 
within  25% 

Static  repeatability 

Response  to  standard  test 
object 

Twice-daily  static 
measurements  of 
standard  object 

Response:  +/-  10% 

Spatial  coverage  in 
detection  survey 

Extended  footprint  coverage 

Mapped  survey  data 

98%  coverage 

Detection  of  all 
targets  of  interest 
(TOI) 

Percent  detected  of  seeded 
anomalies 

Location  of  seeded 
items 

Anomaly  list 

100%  of  seeded  items 
detected  with  0.6-m 
halo 

Static  Data  Collection  Objectives 

Repeatability  of 

IVS  measurements 

Percentage  of  IVS  items 
identified  correctly 

Twice-daily  IVS  survey 
data 

98%  of  IVS  items 
identified  correctly 
with  a  confidence 
metric  of  >  0.70 

Production  rate 

Number  of  cued 
interrogations  per  day 

Pre-processing  time 

Log  of  field  work  and 
data  pre-processing  time 

100  anomalies  per  day 
Pre-processing  time 
<3  min  per  target 

Cued  interrogation 
of  anomalies 

Instrument  position 

Cued  data 

100%  of  anomalies 
where  the  center  of  the 
cued  pattern  is 
positioned  within  40 
cm  of  modeled  source 
location 
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Table  3-1:  Performance  Objectives  for  This  Demonstration  (cont.) 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Analysis  and  Classification  Objectives 

Maximize  correct 
classification  of 

TOI 

Number  of  TOI  retained 

Ranked  anomaly  lists 
Scoring  reports  from 
Institute  for  Defense 
Analyses  (IDA) 

Approach  correctly 
classifies  all  TOI 

Maximize  correct 
classification  of 
non-  TOI 

Number  of  false  alarms 
eliminated 

Ranked  anomaly  lists 
Scoring  reports  from 

IDA 

Reduction  of  clutter 
digs  required  by  >75% 
while  retaining  all  TOI 

Specification  of  no¬ 
dig  threshold 

Probability  of  correct 
classification  of  TOI  and 
number  of  false  alarms  at 
demonstrator  operating 
point 

Demonstrator-specified 

threshold 

Scoring  reports  from 

IDA 

Threshold  specified  by 
the  demonstrator  to 
achieve  criteria  above 

Minimize  number 
of  anomalies  that 
cannot  be  analyzed 

Number  of  anomalies  that 
must  be  classified  as 
“Cannot  Analyze” 

Demonstrator  target 
parameters 

Reliable  target 
parameters  can  be 
estimated  for  >  95%  of 
anomalies  on  the 
detection  list. 

Correct  estimation 
of  target  parameters 

Accuracy  of  estimated 
target  parameters  for  seed 
items 

Demonstrator  target 
parameters 

Results  of  intrusive 
investigation 

Polarizabilities  ±  20% 
X,Y  <15cm(lG) 

Z  <10  cm  (la) 

3.1  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  STRIP  MEASUREMENTS 
/  STATIC  REPEATABILITY 

The  reliability  of  the  survey  data  depends  on  the  proper  functioning  of  the  survey  equipment. 
This  objective  concerns  the  twice-daily  confirmation  of  sensor  system  performance.  The 
repeatability  of  instrument  verification  strip  (IVS)  measurements  was  originally  specified  as  the 
test  to  measure  dynamic  instrument  functionality;  however,  the  variable  geology  at  the  site,  the 
lack  of  consistently  flat  areas  in  which  to  place  the  IVS  items  in  each  area  of  interest  (AOI),  and 
the  relatively  short  amount  of  time  spent  in  each  AOI  (small  number  of  repeated  measurements 
over  each  IVS  with  which  to  compare  single  tests)  were  not  conducive  to  the  collection  of 
consistent  responses  over  each  item.  To  prove  that  the  EM61  used  for  the  dynamic  collection 
was,  in  fact,  functional  each  day,  a  static  repeatability  test  was  substituted  for  the  IVS  test. 

3.1.1  Metric 

The  metric  for  the  static  repeatability  test  was  the  measured  response  for  a  small  bolt  versus  a 
standard  value  determined  for  the  bolt  over  the  course  of  the  project. 
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3.1.2  Data  Requirements 

Twice  daily  measurements  of  the  response  generated  by  the  bolt  when  it  was  placed  in  a 
consistent  location  relative  to  the  EM61  coil  were  used  to  judge  this  objective. 

3.1.3  Success  Criteria 

This  objective  was  met  if  the  static  response  for  the  bolt  was  within  10%  of  a  standard,  expected 
response.  The  standard  response  for  the  bolt  was  defined  as  the  average  of  all  of  the 
measurements  collected  for  this  item  during  the  course  of  the  dynamic  IVS  surveys. 

3.2  SPATIAL  COVERAGE 

The  detection  survey  was  intended  to  cover  a  maximum  of  the  AOI  so  that  all  detectable  targets 
were  detected.  Targets  are  detectable  if  the  transmitted  field  is  sufficiently  strong  to  reach  the 
target  and  if  the  measured  target  response  is  sufficiently  strong  in  return  to  exceed  a  given 
threshold. 

3.2.1  Metric 

The  metrics  for  this  objective  were  collected  coverage  area  versus  between-line  gaps  larger  than 
70  centimeters  (cm)  and  versus  between-line  gaps  large  enough  that  there  was  a  considerable  risk 
that  a  TOI  might  be  missed  completely  (90  cm). 

3.2.2  Data  Requirements 

The  percentage  of  the  coverage  area  with  between-line  gaps  larger  than  70  cm  and  90  cm  were 
calculated  for  each  day’s  data  using  the  footprint  coverage  tool  in  Oasis  montaj’s  UX-Detect 
tool. 

3.2.3  Success  Criteria 

This  objective  was  met  if  more  than  98%  of  each  day’s  coverage  area  had  a  line  spacing  of 
70  cm  or  less  and  if  there  were  no  between-line  gaps  of  greater  than  90  cm. 

3.3  DETECTION  OE  ALL  TARGETS  OF  INTEREST 

Quality  detection  data  should  lead  to  a  high  probability  of  detecting  all  targets  of  interest  at  the 
site. 


3.3.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  seed  items  that  were  detected  using  the 
specified  anomaly  selection  threshold. 

3.3.2  Data  Requirements 

The  centers  of  all  seed  items  were  measured  using  a  real-time  kinematic  global  positioning 
system  (GPS)  when  they  were  placed  in  the  ground.  Dynamic  target  selections  were  compared  to 
the  known  seed  item  locations  as  dynamic  target  lists  were  submitted,  and  the  horizontal  distance 
was  calculated  between  the  seed  locations  and  the  nearest  dynamic  target  location. 
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3.3.3  Success  Criteria 

The  objective  was  considered  met  for  each  seed  item  if  a  dynamic  target  was  within  60  cm  of  the 
measured  seed  location. 

3.4  CORRECTLY  IDENTIFY  SEED  ITEMS  IN  THE  INSTRUMENT  VERIFICATION 
STRIP 

The  IVS  strip  constructed  at  each  AOI  contained  four  inert  37-mm  seed  items.  MetalMapper  data 
were  collected  over  one  of  the  IVSs  (dependent  on  the  area  being  surveyed)  twice  daily. 

3.4.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  IVS  items  correctly  classified  during  the 
project. 

3.4.2  Data  Requirements 

Daily  IVS  data  were  collected  and  processed  in  the  same  manner  as  all  other  target  points 
acquired  during  the  project.  Following  analysis,  each  IVS  target  was  labeled  with  an  identified 
source  object  and  a  confidence  metric  that  quantified  the  degree  of  match  between  three 
polarizability  curves  generated  from  the  measured  IVS  data  and  three  polarizability  curves  for  a 
similar  item  in  a  target  library. 

3.4.3  Success  Criteria 

The  performance  objective  for  the  project  was  the  correct  classification  of  all  IVS  seed  items 
with  a  confidence  metric  of  0.80  or  higher.  Due  to  the  potential  for  the  occasional  collection  of  a 
poor  data  point  without  the  operator’s  immediate  knowledge,  the  success  criteria  was  slightly 
lower  than  100%,  with  the  project  deemed  successful  if  more  than  98%  of  the  IVS  data  points 
were  classified  correctly. 

3.5  PRODUCTION  RATE 

This  objective  corresponds  to  data  collection  and  pre-processing  time. 

3.5.1  Metric 

The  metrics  for  this  objective  are  the  mean  daily  survey  rate  and  the  mean  pre-processing 
time  per  anomaly. 

3.5.2  Data  Requirements 

The  number  of  surveyed  anomalies  and  the  pre-processing  time  for  each  were  recorded  daily. 

3.5.3  Success  Criteria 

The  objective  was  considered  met  if  the  mean  daily  survey  rate  is  at  least  100  anomalies  and  if 
pre-processing  time  is  less  than  3  minutes  per  target. 
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3.6  CUED  INTERROGATION  OF  ANOMALIES 

The  reliability  of  cued  data  depends  on  acceptable  instrument  positioning  during  data  collection 
in  relation  to  the  actual  source  location. 

3.6.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  sources  that  were  within  an  acceptable 
distance  of  the  center  of  the  instrument  during  data  collection. 

3.6.2  Data  Requirements 

The  MetalMapper  sensor  location  is  determined  during  the  inversion  of  the  collected  data  and  is 
resolved  using  the  location  identified  by  the  GPS  sensor  directly  over  the  middle  of  the  sensor 
and  pitch  and  roll  data  supplied  by  an  inertial  movement  unit.  The  sensor  location  is  reported  as 
the  X_Array  and  Y_Array  channels  in  the  Geosoft  target  database.  The  location  of  the  source 
object  is  also  calculated  during  target  inversion  and  was  defined  as  the  Fit_X[8]  and  Fit_Y[8] 
channels  in  the  Geosoft  target  database.  The  distance  between  these  two  locations  was  calculated 
for  cued  data  point. 

3.6.3  Success  Criteria 

The  performance  objective  was  for  all  single  object  solver  targets  to  have  modeled  source 
locations  within  40  cm  of  the  center  of  the  sensor  unless  a  re-shot  had  already  been  performed  on 
that  target. 

3.7  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TARGETS  OF  INTEREST 

One  of  the  two  main  objectives  of  this  pilot  study  was  to  show  that  classification  could  correctly 
identify  all  seeded  items  and  any  native  items  that  resembled  ordnance  that  could  be  considered 
MFC  as  TOI.  Native  items,  including  practice  warheads,  motors,  and  functioned  or  previously 
detonated  but  intact  high  explosive  warheads,  were  identified  as  TOI  during  the  intrusive 
investigation  if  they  resembled  a  potentially  hazardous  item  to  the  degree  that  not  investigating 
such  items  would  present  a  serious  hazard. 

3.7.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  items  classified  as  TOI  following  the 
intrusive  investigation  that  were  correctly  identified  as  objects  that  should  be  dug  in  the  final 
ranked  dig  list. 

3.7.2  Data  Requirements 

Following  data  collection,  MetalMapper  data  were  analyzed  to  create  a  prioritized  dig  list,  which 
assigned  each  target  to  one  of  four  categories:  1)  TOI  2)  Non-TOI,  3)  Can’t  Analyze,  or  4) 
Training.  The  targets  classified  as  TOI,  Can’t  Analyze,  and  Training  were  considered  “dig” 
targets.  The  list  of  items  identified  as  TOI  following  the  completion  of  intrusive  operations, 
including  seed  items,  was  compared  to  those  targets  marked  “dig”  in  the  ranked  dig  list.  If  any 
training  items  had  been  identified  as  TOI  upon  recovery,  the  item(s)  would  have  been  added  to 
the  classification  library  to  identify  other  similar  sources  at  the  site. 
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3.7.3  Success  Criteria 

The  performance  objective  was  the  correct  identification  of  all  TOI  (including  blind  seed  items) 
as  targets  that  should  be  intrusively  investigated,  or  “dig”  targets.  The  project  was  considered 
successful  if  100%  of  the  TOI  were  labeled  as  “dig”  targets  in  the  final  ranked  dig  list.  No 
distinction  was  made  between  a  target  correctly  identified  as  TOI  and  a  target  identified  as  a 
Can’t  Analyze  point  for  this  objective.  Each  TOI  simply  needed  to  be  indicated  as  a  target  that 
should  be  investigated. 

3.8  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TARGETS  OF  INTEREST 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  In  addition  to  correctly  classifying  TOI,  the  effectiveness  of  the  MetalMapper  in 
discriminating  munitions  is  a  function  of  the  degree  to  which  responses  that  do  not  correspond  to 
TOI  can  be  eliminated  from  consideration  during  the  intrusive  investigation. 

3.8.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  items  classified  as  non-TOI  following  the 
intrusive  investigation  that  were  correctly  identified  as  objects  that  did  not  need  to  be  intrusively 
investigated  in  the  final  ranked  dig  list. 

3.8.2  Data  Requirements 

Following  data  collection,  MetalMapper  data  were  analyzed  to  create  a  prioritized  dig  list,  which 
assigned  each  target  to  one  of  three  categories:  1)  TOI,  2)  Non-TOI,  or  3)  Can’t  Analyze.  The 
targets  classified  as  non-TOI  were  considered  “no  dig”  or  non-TOI  targets.  The  list  of  items 
identified  as  non-TOI  following  the  completion  of  intrusive  operations  was  compared  to  those 
targets  marked  “no  dig”  in  the  ranked  dig  list.  Two  thousand  targets  were  intrusively  investigated 
during  this  project.  Therefore,  all  targets  not  investigated  during  the  intrusive  effort  are 
considered  non-TOI. 

3.8.3  Success  Criteria 

The  performance  objective  was  considered  met  if  more  than  75%  of  the  non-TOI  items  were 
correctly  labeled  as  non-TOI  while  retaining  all  of  the  TOI  (Section  3.7)  above  the  dig  threshold. 

3.9  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE  ANALYZED 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  using  the  MetalMapper  data  cannot 
be  classified.  These  anomalies  must  be  placed  in  the  dig  category,  which  reduces  the 
effectiveness  of  the  classification  process. 

3.9.1  Metric 

The  percentage  of  anomalies  for  which  reliable  parameters  could  not  be  estimated  was  the  metric 
for  this  objective. 

3.9.2  Data  Requirements 

Those  targets  for  which  parameters  could  not  be  reliably  estimated  were  identified  as  such  on  the 
prioritized  dig  list  submitted  following  analysis  of  the  MetalMapper  cued  data. 
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3.9.3  Success  Criteria 

The  performance  objective  was  considered  met  if  reliable  parameters  could  be  estimated  for 
greater  than  95%  of  the  targets  on  the  prioritized  dig  list. 

3.10  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

This  objective  involves  the  accuracy  of  the  modeled  location  for  target  source  objects.  Both  the 
dig  team  and  stakeholders  are  more  confident  that  the  correct  source  object  is  being  investigated 
if  the  estimated  location  is  relatively  close  to  the  recovered  object’s  location,  both  horizontally 
and  vertically. 

3.10.1  Metric 

The  distance  between  the  inverted  target  location  and  the  location  of  the  object(s)  recovered 
during  the  intrusive  investigation  was  the  metric  for  this  objective. 

3.10.2  Data  Requirements 

The  dig  list  submitted  to  the  intrusive  team  contained  X,  Y  locations  for  each  target  as 
determined  during  inversion  of  that  target.  The  dig  teams  measured  the  offset  between  any 
recovered  items  and  the  location  listed  on  the  dig  sheet.  The  offsets  were  compared  following  the 
completion  of  the  project. 

3.10.3  Success  Criteria 

The  project  objective  was  considered  met  if  one  standard  deviation  of  the  distance  between  the 
estimated  X,  Y  locations  and  the  recovery  locations  were  within  15  cm  and  the  estimated  depths 
were  within  10  cm  (1  standard  deviation  [a]). 
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4.0  SITE  DESCRIPTION 


4.1  SITE  DESCRIPTION 

The  former  WMA  is  on  the  northwest  side  of  the  Big  Island  of  Hawaii  between  Waikoloa 
Village  and  Waimea.  The  demonstration  was  conducted  over  three  AOIs:  Task  Order  (TO)  20 
Area  A,  TO20  Area  B,  and  TO  17.  A  map  of  the  demonstration  area  and  AOIs  is  shown  in 
Figures  4-1  through  4-3. 

4.2  SITE  SELECTION 

This  site  was  chosen  as  one  in  a  series  of  sites  for  demonstration  of  the  classification  process. 
Sites,  including  this  one,  provide  opportunities  to  demonstrate  the  capabilities  and  limitations  of 
the  classification  process  on  a  variety  of  site  conditions.  Further  information  about  ESTCP’s 
classification  program  can  be  found  at  http://www.serdp-estcp.org/Featured-Initiatives/Munitions 
-Response-Initiatives/Classification-Applied-to-Munitions-Response.  This  site  was  selected  for 
the  program  because  of  its  terrain  and  an  opportunity  to  involve  a  stakeholder  community, 
including  state  regulators,  in  the  classification  pilot  program. 

4.3  BRIEF  SITE  HISTORY 

The  100,000-acre  former  WMA  FUDS  was  acquired  by  the  Navy  in  1943  and  used  as  a  military 
training  camp  and  artillery  range  for  50,000  troops  until  1945.  Two  surface  clean-up  activities 
were  performed  in  1946  and  1954.  The  1946  clean-up  was  done  after  the  departure  of  the 
military.  The  1954  clean-up  followed  an  accidental  detonation  of  a  dud  fuse  or  shell  that  killed 
two  civilians  and  seriously  injuring  three  others.  Munitions  and  explosives  continue  to  be 
discovered  at  the  former  WMA.  Investigation  and  clearance  continue  in  areas  planned  for 
development  and  where  the  risk  assessments  were  rated  moderate  to  high. 

To  date,  more  than  100  different  types  of  munitions  have  been  found,  including  mortars, 
projectiles,  hand  grenades,  rockets,  land  mines,  and  Japanese  ordnances.  More  than  1,800  MEC 
items,  117,000  pounds  of  military  debris,  and  149,000  pounds  of  munitions  debris  (MD)  have 
been  cleared  from  22,  600  acres  of  the  former  WMA.  The  work  being  performed  at  the  former 
WMA  is  being  performed  under  the  direction  of  the  USAGE  Honolulu  District. 

4.4  MUNITIONS  CONTAMINATION 

The  suspected  munitions  in  the  former  WMA  include: 

•  37-mm  projectiles 

•  60-mm  and  80-mm  high  explosive  mortars 

•  75-mm,  105-mm,  and  155-mm  projectiles 

•  2.36-in  rocket  propelled  anti-tank  rounds 

•  U.S.  Mk  II  hand  grenades 

•  Ml  anti-tank  land  mines 

•  Japanese  ordnance 
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4.5  SITE  GEODETIC  CONTROL  INFORMATION 

The  coordinates  for  the  locations  of  the  existing  control  points  used  for  this  demonstration  are 
provided  in  Table  4-1.  Both  control  points  were  set  by  a  survey  team  from  Environet,  Inc. 
(Environet),  the  contractor  currently  performing  removal  actions  in  TO  17  and  TO20  at  the 
former  WMA. 


Table  4-1:  Geodetic  Control  Locations 


Identification 

Type 

Northing 
(UTM  5N 
WGS84  m) 

Easting 
(UTM  5N 
WGS84  m) 

Ellipsoid  Height  (m) 

CP13A 

Control  Point 

212907.6 

2214682 

547.272 

CP08 

Control  Point 

205252.2 

2211271 

32.215 

4.6  SITE  CONFIGURATION 

Three  data  collection  areas,  TO  17  and  TO20  Areas  A  and  B,  were  selected  for  data  collection 
during  the  project  (Figure  4-1).  The  T017  and  TO20  areas  were  chosen  based  on  the  different 
age  of  the  lava  flows  that  compose  the  bedrock  at  each  and  the  desire  to  evaluate  whether  the  age 
of  the  flow  would  have  any  effect  on  geophysical  background  response.  The  lava  flow 
underlying  the  TO  17  site  near  the  coast  (Figure  4-2)  is  considerably  younger  than  the  flow 
underlying  the  two  TO20  sites  (Figure  4-3).  Within  each  TO  area,  data  collection  locations  were 
generally  chosen  based  on  relatively  flat  topography  that  could  be  traversed  safely  with  the 
MetalMapper. 
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Figure  4-1  Data  Collection  Boundaries 
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Figure  4-2  Data  Collection  Boundary,  T017 
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Figure  4-3  Data  Collection  Boundaries,  TO20  Areas  A  and  B 
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5.0  TEST  DESIGN 


5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  objective  of  this  program  is  to  demonstrate  a  method  for  the  use  of  classification  in  the 
munitions  response  process.  The  three  key  components  of  this  method  are  collection  of  high- 
quality  geophysical  data  and  principled  selection  of  anomalous  regions  in  those  data;  analysis  of 
the  selected  anomalies  using  physics-based  models  to  extract  target  parameters  such  as  size, 
shape,  and  materials  properties;  and  the  use  of  those  parameters  to  construct  a  ranked  anomaly 
list.  The  geophysical  data  collected  over  each  target  were  processed  using  existing  routines  to 
extract  target  parameters.  These  parameters  were  passed  to  the  classification  routines  which, 
after  training  on  a  limited  amount  of  site-specific  ground  truth,  were  used  to  produce  prioritized 
anomaly  lists. 

Validation  digging  was  coordinated  by  the  Program  Office.  Because  this  was  a  demonstration, 
all  anomalies  on  the  master  anomaly  list  were  investigated.  The  underlying  target  was 
uncovered,  photographed,  located  with  a  cm-level  GPS  system,  and  removed.  Each  analysis 
demonstrator  was  able  to  request  ground  truth  data  for  training. 

At  the  conclusion  of  training,  each  demonstrator  submitted  an  initial  ranked  anomaly  list  for  each 
data  set  analyzed.  These  lists  were  ordered  from  the  item  the  demonstrator  was  most  confident 
was  a  munitions  through  the  item  the  demonstrator  was  most  confident  was  not  hazardous  and 
indicated  the  threshold  the  demonstrator  chose  for  initial  digging.  Anomalies  for  which  the 
demonstrator  could  not  extract  meaningful  parameters  were  placed  at  the  top  of  the  list.  The  final 
inputs  were  scored  by  the  Institute  for  Defense  Analyses,  with  emphasis  on  the  number  of  items 
that  are  correctly  labeled  nonhazardous  while  correctly  labeling  all  TOI. 

The  primary  objective  of  the  demonstration  was  to  assess  how  well  each  demonstrator  ordered 
the  ranked  anomaly  list(s)  and  specified  the  threshold  separating  high  confidence  clutter  from  all 
other  items.  The  secondary  objective  was  to  determine  the  classification  performance  that  could 
be  achieved  by  each  approach  through  a  retrospective  analysis. 

5.2  SITE  PREPARATION 

5.2.1  Survey  of  Historical  Records 

Historical  information  on  this  site  has  been  referenced  in  the  stakeholder  review  draft  of  the 
Waikoloa  Site  Inspection  Report,  Army  National  Guard  Munitions  Response  Sites.  This  report 
is  posted  on  the  ESTCP  FTP  (file  transfer  protocol)  server  and  can  be  used  for  reference. 

5.2.2  First-Order  Navigation  Points 

The  two  control  points  used  for  this  project  (Table  4-1)  were  set  by  a  survey  team  from 
Environet.  The  points  were  originally  set  using  a  Universal  Transverse  Mercator  (UTM)  Zone  5 
Hawai’i  North  American  Datum  of  1983  (NAD83)  coordinate  system,  which  is  approximately 
75  cm  offset  from  the  UTM  Zone  5  World  Geodetic  System  (WGS)  1994  datum  used  on  most 
geophysical  projects.  The  points  in  Table  4-1  are  in  UTM  Zone  5  WGS84. 
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5.2.3  EM61  Transect  Survey 

A  transect  survey  was  performed  in  TO20  using  a  single  hand-towed  EM61,  with  the  intention  of 
determining  anomaly  densities  in  areas  that  would  potentially  be  covered  by  later  100%  coverage 
dynamic  surveys  and  cued  MetalMapper  surveys.  The  transect  data  were  helpful  in  showing  that 
anomaly  densities  in  most  of  the  sites  identified  as  possible  locations  for  the  advanced  EMI  data 
collection  were  relatively  low  but  did  not  play  much  of  a  factor  in  determining  exactly  which 
areas  would  be  selected  for  additional  data  collection.  Site  topography  ended  up  being  much 
more  important  in  the  site  selection  process  than  anomaly  density,  although  the  low  densities 
noted  in  the  transect  surveys  did  lead  to  the  “seeding”  of  additional  non-TOI  (scrap  from  areas 
adjacent  to  the  data  collection  area)  in  TO20  Area  B. 

5.2.4  Surface  Clearauce 

A  surface  sweep  was  performed  by  UXO  technicians  from  Environet  before  starting  any 
geophysical  operations.  Typically,  surface  sweeps  are  undertaken  to  remove  larger  metallic 
debris  on  the  ground  surface  that  would  interfere  with  data  collection  or  to  remove  all  metallic 
debris  in  cases  where  the  quantity  of  such  debris  would  significantly  affect  the  collected  data.  In 
the  case  of  the  former  WMA  sites,  anomaly  densities  were  low  enough  that  leaving  debris 
present  on  the  ground  surface  did  not  present  a  concern  with  regard  to  elevated  anomaly 
densities.  Given  the  rocky  nature  of  the  sites  and  the  relatively  thin  soil  layers  present,  it  was 
actually  suspected  that  most  metallic  debris  present  on  site  might  have  been  on  or  very  close  to 
the  surface.  So  the  surface  sweep  performed  for  the  project  was  only  intended  to  clear  the  area  of 
any  surface  munitions  that  would  present  a  possible  hazard  to  the  team  conducting  the  100% 
coverage  dynamic  survey  and  the  cued  MetalMapper  data  collection.  All  nonhazardous  debris 
was  left  in  place. 

5.2.5  Seeding  Operation 

At  a  site  such  as  this,  there  is  usually  a  high  ratio  of  clutter  to  TOI.  To  determine  classification 
performance  with  acceptable  confidence  bounds,  the  site  was  seeded  with  enough  TOI  to  ensure 
reasonable  statistics. 

Parsons’  subcontractor  Environet  conducted  seeding  operations  at  WMA  from  August  28,  2013, 
through  September  4,  2013,  and  on  September  17,  2013.  Medium  and  small  industry  standard 
objects  (ISOs),  inert  37-mm  projectiles,  inert  60-mm  mortars,  and  inert  81-mm  mortars  were 
buried  throughout  the  survey  area  as  quality  control  (QC)  blind  seed  items  before  the  start  of 
detection  survey  data  collection.  The  location  of  each  seed  item  was  established  with  a  Trimble 
R8  real-time  kinematic  GPS  system.  Seeds  consisted  of  7  medium  and  10  small  ISOs,  48  inert 
37-mm  projectiles,  20  inert  60-mm  mortars,  and  12  inert  81-mm  mortars.  The  items  used  as 
seeds  were  borrowed  from  the  USAGE  and  Naval  Research  Laboratory  and  were  returned  at  the 
end  of  the  project. 

Environet  practiced  anomaly  avoidance  at  each  location  for  safety  and  to  ensure  a  clean  area  for 
emplacement.  Despite  hard  ground  conditions  at  the  three  AOIs,  all  97  seeds  were  placed  at  the 
depths  specified  in  the  seeding  plan.  Excavation  of  the  seed  holes  involved  manual  digging  to 
meet  exact  depth/orientation  specifications  and  to  minimize  burial  evidence.  A  photo  was  taken 
of  the  seed  item  at  the  burial  location  and  included  a  whiteboard  showing  all  emplacement 
information. 
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Seed  location  holes  were  not  backfilled  until  final  QC  checks  were  complete.  QC  checks 
consisted  of  comparing  the  location  with  the  original  designated  location;  capturing  the  center 
location  of  the  emplaced  seed  item  with  GPS;  and  checking  the  depth,  inclination,  and  dip  angle 
of  each  seed  item.  After  these  checks  were  complete,  the  hole  was  backfilled  with  a  shovel  to 
prevent  any  excess  movement  of  the  seed  items.  Table  5-1  lists  the  seed  items  emplaced  for  the 
project. 


Table  5-1:  WMA  Demonstration  Seed  Items 


Seed  Item 

Total 

Small  ISO 

10 

Medium  ISO 

7 

37  mm 

48 

60  mm 

20 

81  mm 

12 

Total 

97 

5.2.6  Establish  an  Instrument  Verification  Strip 

A  relatively  clean  area  for  use  as  the  IVS  was  identified  in  or  near  each  AOI.  Based  on  a  lack  of 
available  seed  items,  all  IVS  items  were  the  same  four  inert  37-mm  projectiles  that  were  moved 
from  AOI  to  AOI  as  the  MetalMapper  was  moved.  The  projectiles  were  buried  horizontally  at  a 
depth  of  10  cm  at  approximately  5-m  intervals  in  each  AOI  except  in  TO  17. 

5.3  SYSTEM  SPECIFICATION 

The  MetalMapper  sensor  and  data  acquisition  system  are  described  in  detail  in  Section  2.1. 
During  the  demonstration  study,  the  antenna  array  was  transported  via  a  sled  mounted  to  the 
front  of  an  extendable  reach  forklift  (Figure  2-2).  A  Trimble  R8  GPS  was  mounted  directly 
above  the  sensor  array  using  a  wooden  tripod,  and  an  inertial  measurement  unit  was  attached  to 
the  wooden  support  used  to  stabilize  the  X-  and  Y-direction  transmitters,  also  directly  above  the 
center  of  the  array.  These  instruments  streamed  positional  data  constantly  at  a  rate  of  5  Hz.  The 
two  instruments  were  connected  to  the  DAQ  via  universal  serial  bus  ports.  Incoming  GPS  data 
were  used  both  to  navigate  from  point  to  point  and  to  locate  the  collected  response  data.  Inertial 
movement  unit  corrections  are  not  performed  in  real  time  and  integrated  with  the  incoming  GPS 
data  on  the  MetalMapper  screen,  but  they  were  used  to  correct  the  locations  of  all  collected  GPS 
points  based  on  the  pitch,  roll,  and  yaw  information  recorded  with  the  GPS  measurements. 

5.4  CALIBRATION  ACTIVITIES 

5.4.1  Test  Pit  and  Instrument  Verification  Strip  Data  Collection 

A  test  pit  was  constructed  at  the  site  before  the  start  of  cued  MetalMapper  data  collection.  The 
pit  was  an  approximately  3-foot  by  3-foot  by  3-foot  hole  that  allowed  the  collection  of  static 
MetalMapper  data  over  TOI  items  expected  at  the  site.  The  test  pit  data  could  then  be  used  for 
comparison  with  field  data  collected  over  unknown  targets.  Test  pit  items  were  various  depths  in 
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the  hole  during  collection  to  determine  the  effects  of  depth  on  MetalMapper  response,  and  data 
were  collected  over  the  items  oriented  horizontally  and  vertically  at  each  depth.  In  addition  to  the 
data  collected  over  items  in  the  test  pit,  a  test  stand  was  set  up  to  collect  data  over  smaller 
munitions  that  had  been  found  by  Environet  over  the  course  of  their  work  at  the  site.  As  with  the 
test  pit  data,  test  stand  data  were  collected  with  the  items  at  various  distances  below  the  sensor 
and  at  different  orientations.  Test  pit  data  were  collected  on  August  29  and  30,  and  test  stand 
data  were  collected  on  September  6.  The  items  tested  and  the  depths  and  orientations  used  for 
testing  are  indicated  in  Table  5-2. 


Table  5-2:  MetalMapper  Test  Pit/Test  Stand  Data  Collected 


Item  Identification 

Depths  (cm) 

Orientations 

Test  pit  data  (8/29/13  and  8/30/13) 

37-mm  projectile,  4.5-in  long, 

0,  10,  20,  25 

Horizontal,  vertical 

rotating  band 

(each  depth  except  surface  [horizontal  only]) 

37-mm  projectile,  3. 5 -in  long. 

0,  10,  20,  30 

Horizontal,  vertical 

rotating  band 

(each  depth  except  surface  [horizontal  only]) 

37-mm  projectile,  4.5-in  long. 

0,  10,  20,  30 

Horizontal,  vertical 

no  rotating  band 

(each  depth  except  surface  [horizontal  only]) 

60-mm  mortar,  warhead  only,  training 

0,  10,  30,  40 

Horizontal,  vertical 

(each  depth  except  surface  [horizontal  only]) 

60-mm  mortar,  with  tail 

0,  10,  30,  40 

Horizontal,  vertical 

(each  depth  except  surface  [horizontal  only]) 

60-mm  mortar,  9.25-in,  hollow. 

0,  10,  25 

Horizontal,  vertical 

with  tail 

(each  depth  except  surface  [horizontal  only]) 

60-mm  mortar,  7-in,  hollow,  with  tail 

0,  10,  30 

Horizontal,  vertical 

(each  depth  except  surface  [horizontal  only]) 

60-mm  mortar,  5 -in,  hollow,  no  tail 

0,  10,  30 

Horizontal,  vertical 

(each  depth  except  surface  [horizontal  only]) 

81 -mm  mortar,  with  tail 

0,  25 

Horizontal,  vertical,  60  degrees  nose  up,  and 

60  degrees  nose  down 

Test  stand  data  (9/6/13) 

M48A2  fuze 

16.5,20 

Horizontal,  vertical  (nose  up)  (each  depth) 

M48  unknown  mod  fuze 

16.5,20 

Horizontal,  vertical  (nose  up)  (each  depth) 

M54  time  fuze,  superquick  element 

16.5,  20 

Horizontal,  vertical  (nose  up)  (each  depth) 

M58  37-mm  base  fuze 

16.5,20 

Horizontal,  vertical  (nose  up)  (each  depth) 

M63  37-mm  high  explosive 

16.5 

Horizontal 

projectile,  version  1 

20 

Vertical 

25 

Horizontal 

M63  37-mm  high  explosive 

16.5 

Horizontal 

projectile,  version  2 

20 

Vertical 

25 

Horizontal 

In  addition  to  the  test  pit  and  test  stand  data,  data  were  collected  over  the  IVS  twice  daily  during 
the  cued  survey.  All  data  collected  over  the  IVS  strip  were  processed  as  described  in  Section  6.2 
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and  compared  to  either  expected  responses  (EM61)  or  the  WMA  target  library  (cued).  The 
following  tests  were  performed  for  the  collected  I  VS  data: 

•  Dynamic:  It  was  intended  that  dynamic  responses  measured  over  seed  items  in  the  I  VS 
strip  be  within  25%  of  the  expected  response  for  each  item.  However,  due  to  site 
conditions  (i.e.,  geology,  rocky  terrain)  and  because  the  data  collection  crew  was  never  at 
an  AOI  for  more  than  a  couple  of  days,  the  IVS  surveys  proved  to  be  too  inconsistent  to 
pass  this  metric.  IVS  items  were  consistently  detected  with  responses  that  would  have  led 
to  their  selection  as  targets  in  a  dynamic  data  set,  but  over  the  course  of  the  relatively  few 
passes  collected  over  the  IVS  at  each  site,  they  were  generally  not  all  within  25%  of  the 
average  value  for  each  IVS.  It  is  suspected  that  small  changes  in  the  path  over  the  items 
and  the  widely  variable  geology  at  the  site  were  the  major  causes  of  inconsistent 
responses.  Daily  static  tests  were  substituted  for  the  IVS  testing  to  ensure  that  the  EM6I 
used  for  the  dynamic  data  collection  was  functional  and  that  readings  were  consistent 
over  the  course  of  the  project.  The  static  test  involved  the  collection  of  1  minute  of  data 
with  no  source  present  and  1  minute  of  data  with  a  small  bolt  placed  in  the  center  of  the 
EM61  coil.  The  mean  response  for  the  test  with  the  bolt  subtracted  from  the  mean 
background  response  to  determine  the  response  caused  by  the  bolt.  All  responses 
measured  during  the  project  were  averaged  to  determine  an  expected  response  for  the 
bolt,  and  each  individual  measured  response  was  expected  to  be  within  +/-  10%  of  the 
expected  response  to  ensure  that  the  system  was  in  working  order.  As  long  as  the  EM61 
was  not  moved  between  the  background  and  bolt  tests,  this  ensured  that  geology  and 
bumps  over  rocks  did  not  alter  the  measured  response. 

•  Static:  The  item  identified  by  the  target  library  comparison  was  compared  to  the  actual 
buried  item,  and  it  was  expected  that  the  identified  item  matched  TOI  with  a  confidence 
high  enough  that  it  would  be  marked  as  a  dig  (0.7  confidence  >  98%  of  the  time). 

IVS  testing  results  are  detailed  in  Section  7.1. 

5.4.2  Background  Data 

Background  data  were  generally  collected  every  2  hours  throughout  the  project.  MetalMapper 
background  collection  points  were  determined  by  the  operator  who  searched  for  a  clear  location 
using  the  “dancing  arrows”  display  on  the  computer  screen.  Although  this  has  been  an  acceptable 
way  to  collect  background  data  at  other  sites,  it  was  a  more  questionable  practice  at  WMA  given 
the  magnetic  geology  at  the  site  that  produced  a  much  more  significant  response  than  typically 
seen  at  other  demonstration  sites.  In  addition  to  a  the  magnitude  of  the  response,  it  was  also 
highly  variable  across  the  site,  with  lateral  changes  of  a  few  feet  or  vertical  changes  of  a  few 
centimeters  resulting  in  significantly  different  background  values.  Vertical  changes,  or  the 
difference  between  the  height  of  the  instrument  above  the  ground  surface  during  collection  of  a 
background  location  (typically  closer  to  1 5  cm  or  the  height  of  the  sensor  when  placed  on  flat 
ground),  were  actually  fairly  common  given  the  rocky  nature  of  the  site,  because  the  instrument 
could  not  be  placed  directly  on  the  ground  surface  at  target  collection  locations.  Given  the 
extremely  variable  geology  at  the  site,  two  background  points  were  collected  at  each  background 
location,  one  on  the  ground  surface  and  one  approximately  3  feet  in  the  air,  assuming  that  being 
able  to  remove  the  effects  of  consistent  environmental  factors  (e.g.,  forklift,  electronics, 
atmospheric  conditions)  might  prove  useful  rather  than  having  to  depend  on  inconsistent 
background  conditions. 
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BTG  dealt  with  the  variable  background  data  in  different  ways.  The  data  were  corrected  using 
typical  methods  with  the  background  collected  at  the  site,  and  inverted  as  described  in  Section 
6.2.  In  addition,  given  the  uncertainty  of  local  background  measurements,  BTG  also  developed  a 
background  model  based  on  correction  of  the  on-ground  background  data  using  the  in-air 
background  data.  The  model  treated  the  background  response  as  a  plate-like  target  at  a  95-cm 
depth  directly  below  the  middle  of  the  sensor.  As  with  the  data  corrected  using  the  site 
backgrounds,  BTG  performed  various  inversions  for  each  target.  One  of  these  was  a  three-source 
inversion  in  which  one  of  the  three  sources  was  constrained  to  be  its  model  of  background.  BTG 
also  performed  an  un-background-corrected  two-source  inversion  in  which  it  was  assumed  that 
one  of  the  two  modeled  sources  would  be  local  background,  with  the  other  being  the  selected 
EM61  source. 

5.5  DATA  COLLECTION  PROCEDURES 

5.5.1  Dynamic  Data 

Dynamic  data  were  collected  by  hand-pulling  or  carrying  (stretcher  mode)  the  EM61  across  the 
project  site  along  parallel  lines  spaced  50  cm  apart.  The  EM61  had  a  GPS  antenna  directly  over 
the  center  of  the  sensor,  which  transmitted  real-time  positioning  data  to  the  data  logger.  The 
EM61  GPS  data  at  TO20  Area  B  were  collected  using  the  NAD83,  Hawaii  datum,  which  is 
significantly  different  from  the  WGS84  datum  automatically  assumed  by  the  MetalMapper, 
which  led  to  a  significant  shift  (~75  cm)  between  the  target  locations  as  shown  on  the 
MetalMapper  screen  during  collection  and  the  actual  source  locations  in  the  ground.  Sources 
were  generally  to  the  southwest  of  the  indicated  location,  and  the  MetalMapper  team  generally 
started  looking  for  the  source  for  a  particular  target  to  the  southwest  of  the  on-screen  location  at 
this  site.  A  modified  intrusive  process  was  developed  for  TO20  Area  B,  as  described  in  Section 
5.6,  to  ensure  that  no  metallic  sources  were  missed  as  a  result  of  the  datum  shift.  The  datums 
were  synchronized  for  the  TO20  Area  A  and  TO  17  data  collection. 

5.5.2  Cued  Data 

The  operator  moved  the  array  by  lifting  the  sled,  navigating  to  the  vicinity  of  each  selected  point 
using  the  graphic  display  on  the  computer  monitor,  and  setting  the  MetalMapper  down  on  the 
point.  Reacquisition  of  the  EM61  targets  selected  for  cued  data  collection  was  accomplished 
using  dancing  arrows  display  on  the  monitor.  This  display  shows  the  seven  receivers  in  the  array, 
arranged  as  they  are  in  the  Z-coil,  typically  with  a  blue  arrow  pointing  out  of  each.  The  arrows 
point  toward  the  metallic  source  nearest  each  of  the  receivers.  Under  ideal  conditions,  there  is 
one  source  in  the  vicinity  of  the  selected  point,  and  all  of  the  arrows  point  inward  toward  the 
center  of  the  array.  In  the  case  of  multiple  sources,  one  or  more  of  the  outer  arrows  may  point 
outward  from  the  array  toward  another  piece  of  metal.  Generally,  the  operator  attempted  to 
position  the  array  such  that,  at  least,  the  arrows  in  the  three  receivers  closest  to  the  middle  of  the 
coil  were  pointing  at  each  other.  After  the  MetalMapper  was  positioned  correctly  above  the 
target,  the  operator  collected  a  data  point  using  the  settings  indicated  in  Table  5-3. 


23 


Table  5-3:  Cued  Data  Acquisition  Parameters 


Mode 

Tx 

Mode 

Hold-Off 
Time  (4s) 

Block 

Period(s) 

Rep 

Fctr 

Dec 

Fctr 

(%) 

Stk 

Const 

Base 

Freq 

(Hz) 

Decay 
Time  (ps) 

No. 

Gates 

Sample 
Period  (s) 

Sample 
Rate  (S/s) 

Static 

ZYX 

50 

0.9 

27 

10 

10 

30 

8328 

50 

9 

N/A 

An  in-field  QC  program  developed  by  BTG  was  also  used  during  the  project  to  check  on  the 
location  accuracy  of  the  collected  point.  The  QC  program  performed  a  rapid  inversion  of  the 
collected  data  and  identified  the  inverted  source  location  relative  to  the  collection  point  on  the 
MetalMapper  screen.  If  the  inverted  location  was  more  than  30  cm  from  the  collection  point,  the 
operator  moved  the  sensor  over  the  inverted  source  location  and  collected  another  point. 

Static  targets  were  identified  according  to  the  identification  (ID)  determined  for  each  target 
picked  in  the  dynamic  EM6I  survey.  For  repeated  measurements  taken  based  on  the  results  of 
the  in-field  inversion,  10,000  was  added  to  the  original  ID  (e.g.,  the  re-shot  for  0001  was  10001). 
For  repeated  measurements  taken  based  on  a  comparison  of  data  collection  location  to  inverted 
source  location  in  following  data  processing,  50,000  or  60,000  (for  multiple  re-shots)  was  added 
to  the  original  ID. 

5.5.3  Scale  of  Demonstration 

Dynamic  data  were  collected  over  5.3  acres  of  the  WMA  site.  A  total  of  1,048  targets  were 
identified  in  the  dynamic  data  for  further  evaluation  via  the  cued  survey.  The  cued  survey 
covered  1,288  data  points  for  1,032  targets.  The  remaining  16  targets  were  not  collected  due  to 
time  constraints,  although  Parsons  did  ensure  that  data  were  collected  over  all  seed  items  before 
departing  the  site.  The  256  extra  points  were  re-shots  of  already  collected  points  due  to  a  high 
offset  between  the  location  of  the  center  of  the  MetalMapper  and  the  location  of  the  inverted 
source  determined  either  using  the  in-field  QC  program  (>  30-cm  offset)  or  processed  data  (> 
40-cm  offset). 

5.5.4  Sample  Density 

One  data  point  was  collected  per  target,  as  described  in  Section  5.5;  re-shots  were  collected  for 
targets  with  initial  collection  locations  greater  than  40  cm  from  the  inverted  source  location. 

5.5.5  Data  Quality  Checks 

An  instrument  calibration  check  was  conducted  a  minimum  of  twice  a  day  (at  the  beginning  and 
the  end  of  the  field  day)  at  the  IVS.  These  checks  are  performed  to  ensure  that  the  MetalMapper 
is  functional,  properly  calibrated,  and  stable.  Failure  with  regard  to  the  performance  objective 
(Section  3.4)  generally  indicates  a  problem  with  either  the  MetalMapper  and  its  associated 
hardware/software  or  the  data  collection  method  for  the  failing  point.  As  discussed  in  more  detail 
in  Section  7.4,  given  the  varying  background  conditions  at  this  particular  site,  IVS  failures  were 
not  considered  a  sign  of  a  problem  with  either  the  equipment  or  the  data  collection  process. 

An  in-field  check  on  the  data  collection  location  versus  modeled  source  location  was  also 
performed  for  each  data  point  collected.  This  was  accomplished  using  a  program  that  performed 
a  rapid  single  object  inversion  of  the  collected  point  to  determine  a  source  location.  Points  with 
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sources  modeling  farther  than  30  cm  from  the  initially  collected  point  were  re-colleeted  at  the 
indieated  inverted  source  location. 

5.5.6  Data  Handling 

Data  were  recorded  in  binary  format  as  fdes  on  the  hard  disk  of  the  MetalMapper  DAQ.  These 
data  were  offloaded  to  other  media  at  least  once  per  day.  The  computer’s  hard  disk  had  enough 
eapaeity  to  store  all  data  from  the  entire  site,  so  these  data  were  not  erased  until  they  had  been 
thoroughly  reviewed  and  arehived.  The  data  file  names  acquired  each  day  were  eataloged  and 
integrated  with  any  notes  or  eomments  in  the  operator’s  field  book.  All  data  ended  up  on  the  hard 
drives  of  one  or  more  laptop  eomputers  and  were  also  arehived  to  a  data  server  in  the  Parsons 
offiee.  All  .tern  files  colleeted  were  transferred  via  FTP  for  proeessing  by  BTG. 

5.6  INTRUSIVE  PROCEDURES 

Parsons’  intrusive  operations  at  WMA  began  on  February  3,  2014,  and  ended  on  February  13, 
2014.  Operations  began  with  the  site-specifie  training,  which  included  prepping  the  staging  area 
for  intrusive  aetivities  and  performing  equipment  ehecks.  All  Parsons  intrusive  equipment  was 
stored  off  site  in  a  secure  location  that  was  locked  at  the  end  of  the  day.  Daily  equipment  cheek 
ineluded  eonfirming  GPS  aceuraey  over  known  control  points,  EM61  static  tests,  and  handheld 
analog  instruments  Minelab  Explorer  SE  ealibrations. 

The  intrusive  investigation  was  performed  before  submittal  of  a  full  ranked  dig  list,  although 
BTG  did  perform  an  initial  determination  regarding  whether  it  was  more  likely  that  the  dynamie 
anomaly  for  a  given  target  was  due  to  an  aetual  metallic  source  or  to  changing  background 
conditions.  In  addition,  due  to  the  75-em  diserepancy  between  the  two  loeation  datums  used 
during  the  projeet  (NAD83  Hawaii  and  WGS84),  there  was  some  concern  that  the  MetalMapper 
points  collected  in  the  TO20  Area  B  AOl  might  not  have  been  eolleeted  over  the  intended 
dynamie  target.  To  ensure  that  both  the  dynamic  target  source  and  the  sources  classified  by  the 
MetalMapper  (i.e.,  whatever  was  under  the  MetalMapper  during  eolleetion  whether  it  was  the 
dynamie  target  or  not)  both  targets  were  added  to  the  dig  list  if  they  were  more  than  50  cm  apart 
and  the  source  identified  by  the  MetalMapper  did  not  appear  to  be  geologie  in  nature.  The  type 
of  point  to  be  investigated,  EM61  dynamie  target  or  MetalMapper  target,  was  identified  by 
suffixes  added  to  the  target  ID.  Targets  with  an  “A”  suffix  were  MetalMapper  targets  and 
indicated  sourees  that  appeared  to  be  eaused  by  a  metallie  objeet  rather  than  geology.  “A”  targets 
were  not  generated  for  a  given  target  ID  if  the  MetalMapper  data  indicated  that  there  was  no 
metallic  source  present  in  the  collection  location.  Targets  with  a  “B”  suffix  were  EM61  dynamie 
target  locations  that  were  either  more  than  50  em  from  a  metallie  MetalMapper  souree  or 
dynamie  target  locations  where  the  MetalMapper  data  did  not  indicate  the  presenee  of  a  metallie 
object.  “B”  targets  were  not  generated  if  the  dynamic  target  location  was  within  50  cm  of  the 
loeation  of  a  metallie  source  per  the  MetalMapper  data.  Any  suffixes  above  “B”  (e.g.,  “C”,  “D”) 
were  MetalMapper  repeat  shots  that  indicated  the  presence  of  a  metallie  objeet  different  from  the 
one  present  in  the  “A”  location. 

All  intrusively  investigated  anomalies  were  documented  per  the  demonstration  work  plan.  Seed 
items  intrusively  investigated  were  stored  in  a  separate  bin  and  inventoried  daily.  After  all  the 
seed  items  were  accounted  for,  they  were  shipped  off  site. 

Personnel  on  site  to  eonduet  the  intrusive  operation  included  Parsons  and  Environet,  the  UXO 
explosives  subeontraetor.  The  field  team  consisted  of  a  senior  UXO  supervisor,  a  site  safety  and 
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health  manager,  four  Parsons  personnel,  and  two  Environet  personnel.  Parsons’  site  safety  and 
health  manager  and  the  Parsons’  site  manager  conducted  daily  site  safety  briefings,  as 
appropriate. 

5.6.1  Equipment 

The  equipment  used  during  the  former  WMA  intrusive  activities  included  the  following: 

•  Minelab  Explorer  SE 

•  EM61-MK2 

•  Trimble  R8  GPS  system 

•  Miscellaneous  hand  tools 

•  Digital  cameras 

5.6.2  Field  Procedures 

Reacquisition  of  all  targets  was  conducted  using  the  Trimble  R8  GPS  system.  For  TO20  targets, 
the  GPS  base  station  was  set  up  on  survey  monument  CP13A  and  checked  daily  on  monument 
POlSchin.  For  TO  17  targets,  the  GPS  base  station  was  set  up  on  survey  monument  CP012  and 
checked  daily  on  monument  CP8.  Parsons  flagged  all  target  locations  with  a  metal  pin  flag 
marked  with  the  target  identification  and  EM61  pre-value.  The  Minelab  Explorer  SE  was  used  to 
determine  the  initial  approach  to  every  target  and  as  a  screening  process  to  determine  if  metal 
was  present  in  the  subsurface  or  if  the  anomaly  was  caused  by  the  local  geology.  If  the  Minelab 
Explorer  SE  indicated  that  there  was  no  metal  present  in  the  area,  the  anomaly  was  considered  a 
“no  contact.”  A  GPS  point  was  then  taken  at  the  location  of  the  flag  and  a  photograph  was 
collected  of  the  area  surrounding  the  flag.  If  the  Minelab  Explorer  SE  indicated  that  there  was 
metal  present  in  the  subsurface,  the  UXO  technicians  would  carefully  excavate  the  item. 
Location  data  captured  by  GPS  were  used  to  document  the  center  mass  and  depth  of  each  item.  A 
photograph  was  collected  of  the  item  with  written  dig  result  data  on  a  whiteboard.  Lastly,  an 
EM61  unit  was  used  to  scan  the  location  to  confirm  the  absence  of  all  metallic  items  from  that 
target  location  or  that  the  pre-millivolt  reading  had  been  reduced  by  at  least  75%. 

The  Parsons  team  leader  who  orchestrated  the  movements  of  the  different  tasks  associated  with 
the  information-gathering  process  recorded  all  documentation  on  a  dig  sheet.  The  intrusive 
operations  consisted  of  one  intrusive  team  that  was  responsible  for  reacquisition,  intrusive,  and 
anomaly  documentation.  After  enough  target  locations  were  flagged,  the  intrusive  team  would 
split  into  two  teams  to  expedite  the  process. 

All  seed  items  recovered  from  intrusive  operations  were  stored  in  a  secure  area  and  prepared  for 
final  shipment.  Twenty-eight  inert  37-mm  projectiles  and  six  inert  60-mm  mortars  were  shipped 
on  February  25,  2014,  to  U.S.  Army  Engineering  &  Support  Center  in  Huntsville,  Alabama. 
Twenty- five  inert  37-mm  projectiles,  fifteen  inert  60-mm  mortars,  and  twelve  inert  81 -mm 
mortars  were  shipped  on  February  25,  2013,  to  the  Army  Research  Lab  in  Welcome,  Maryland. 

Excavated  holes  greater  than  30  cm  deep  were  left  open  for  the  site  archaeologist  to  investigate 
further  if  need  be.  All  other  target  locations  were  backfilled  after  completion  of  the  excavation. 
After  the  final  anomalies  were  excavated  and  backfilled,  Parsons  conducted  a  walkthrough  and 
confirmed  that  all  holes  were  filled  and  no  trash  was  left. 
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Excavation  data  collected  by  the  intrusive  team  was  digitally  downloaded  to  a  database  and 
reviewed  daily.  The  daily  information  required  the  target  ID  to  be  connected  with  intrusive 
documentation,  photo,  and  GPS  coordinates.  Assessment  of  each  target  item  required  the 
coordinates  to  match  the  original  location  and  the  picture  to  match  the  documented  findings. 
Results  of  the  intrusive  investigation  are  shown  in  Table  5-4.  Photographs  of  the  intrusive 
operation  are  shown  in  Figure  5-1. 


Table  5-4:  Intrusive  Results 


Type 

Items 

Percent 

TO20A  (1.7  acres) 

Other  Debris 

3 

0.8 

Munitions  Debris 

90 

24.6 

No  Contact 

244 

66.6 

Seed 

29 

8.0 

Total 

366 

100 

TO20B  (2.4  acres) 

Other  Debris 

630 

53.9 

Munitions  Debris 

453 

38.8 

No  Contact 

50 

4.3 

Seed 

35 

3.0 

Total 

1,168 

100 

T017  (1.2  acres) 

Other  Debris 

3 

0.9 

Munitions  Debris 

45 

13.0 

No  Contact 

258 

74.6 

Seed 

40 

11.5 

Total 

346 

100 

Overall  Results  (5.3  acres) 

Other  Debris 

636 

33.8 

Munitions  Debris 

588 

31.3 

No  Contact 

552 

29.4 

Seed 

104 

5.5 

Total 

1,880 

100 

5.7  MUNITIONS  DEBRIS  MANAGEMENT 

MD  and  other  debris  (OD)  scrap  recovered  from  the  demonstration  area  at  WMA  amounted  to 
more  than  310  pounds.  The  MD/OD  scrap  was  stored  in  a  locked  storage  area.  Parsons’  senior 
UXO  and  site  safety  officer  certified  all  MD  scrap  by  thoroughly  going  through  each  piece 
individually  before  scrap  was  transferred  to  Environet  for  final  disposition. 
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Figure  5-1:  Intrusive  Operation  Photos 
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6.0  DATA  ANALYSIS  AND  PRODUCTS 


The  MetalMapper  was  used  to  eolleet  statie  data  for  1,031  targets  identified  at  WMA  based  on 
the  5.3  acres  of  EM61  data.  The  processing  and  analysis  steps  that  were  used  to  generate  a  dig/no 
dig  decision  for  each  target  are  described  below. 

6.1  DYNAMIC  DATA 

The  geophysicist  processing  the  EM61  data  merged  the  positioning  and  geophysical  sensor  data 
to  assign  positions  for  each  data  point.  The  located  data  points  were  exported  to  an  ASCII  format 
file,  which  was  then  imported  into  the  Geosoft  Oasis  montaj  geophysical  data  processing 
environment.  Once  in  Oasis  montaj,  the  data  from  the  third  time  gate  were  leveled  using  a  150- 
point  median  statistics  filter.  The  leveled  data  were  gridded  and  displayed  on  a  map,  latency 
corrected  to  remove  any  evident  chevron  patterns,  then  re-gridded  and  displayed  for  target 
selection. 

Targets  were  selected  using  the  Blakely  test  algorithm  in  the  UX-Detect  package  in  Oasis  montaj 
with  a  threshold  of  5  millivolts.  The  Blakely  test  selections  were  then  reviewed  by  the  processor, 
who  merged  target  picks  on  the  same  anomaly  or  added  picks  to  unpicked  peaks  evident  on  the 
color-shaded  grid  map.  After  the  target  list  for  each  dataset  was  finalized,  the  targets  were 
forwarded  to  the  MetalMapper  team  for  cued  collection. 

6.2  CUED  DATA 

All  cued  data  processing,  analysis,  and  classification  was  performed  by  BTG.  Appendix  A 
contains  a  BTG  report  detailing  the  analysis  process  for  the  WMA  cued  data.  Appendix  B 
contains  a  BTG  memorandum  primarily  focused  on  seed  items  incorrectly  classified  as  either 
clutter  or  soil,  although  it  also  briefly  describes  the  process  used  to  compile  the  dig  list  submitted 
for  the  site. 


30 


7.0  PERFORMANCE  ASSESSMENT 


7.1  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  STRIP  MEASUREMENTS 

As  discussed  in  Sections  3.1  and  5.4.1,  the  planned  dynamic  testing  with  the  EM61  proved 
ineffective  given  site  conditions  (i.e.,  geology,  terrain,  lack  of  time  spent  in  each  AOl).  Twice- 
daily  static  tests  were  substituted  for  the  I  VS  testing  to  ensure  that  the  EM61  used  for  the 
dynamic  data  collection  was  functional  and  that  readings  were  consistent  over  the  course  of  the 
project.  A  small  bolt  was  placed  in  the  center  of  the  EM61  coil  for  this  test,  and  all  measured 
responses  for  the  bolt  were  expected  to  be  within  10%  of  the  average  response  calculated  over 
the  entire  project.  All  measured  responses  were  within  3.5%  of  the  average,  indicating  that  the 
EM61  maintained  consistent  readings  throughout  the  project. 

Given  the  troubles  with  the  dynamic  testing  performed,  it  would  make  sense  to  pre-plan  a 
standard  I  VS  location  for  any  future  dynamic  work  conducted  at  the  former  WMA.  The  I  VS 
should  be  located  in  a  flat  area  free  of  rocks  and  accessible  consistently  at  the  start  and  end  of 
each  day  rather  than  depending  on  where  the  field  team  is  working  on  a  given  day.  Ideally,  an 
area  with  relatively  constant  geologic  response  could  be  identified,  although  responses  for  seed 
items  would  likely  be  consistent  in  changing  geologic  conditions  as  long  as  the  EM61  is  not 
bouncing  over  rocks  during  collection. 

7.2  SPATIAL  COVERAGE 

Most  of  the  dynamic  datasets  passed  the  primary  coverage  objective  and  covered  more  than  98% 
of  their  respective  areas  with  a  70-cm  or  less  spacing.  The  notable  exception  was  on  the  west 
side  of  TO20  Area  B,  where  there  were  a  significant  number  of  gaps  due  to  a  faulty  GPS.  Had 
the  dynamic  survey  been  performed  with  the  MetalMapper  as  intended,  these  gaps  would  have 
been  filled  in  to  achieve  the  desired  coverage.  However,  following  the  change  from  the 
MetalMapper  to  the  EM61  for  dynamic  data  collection,  proving  the  technology  for  metrics  that 
would  potentially  be  used  for  future  advanced  sensor  dynamic  surveys  became  less  important. 
More  emphasis  was  placed  on  collecting  enough  dynamic  data  to  identify  1 ,000  discrete  targets 
for  the  follow-on  cued  survey  than  spending  time  filling  in  gaps.  Additional  EM61  data  were 
collected  to  fill  in  gaps  over  seed  items  as  necessary  and  as  possible.  Even  with  the  gap  fills, 
some  seeds  were  buried  so  far  outside  of  the  final  dynamic  survey  areas  that  it  was  extremely 
impractical  to  collect  any  data  over  them.  In  these  cases,  the  seeds  were  simply  added  to  the  cued 
collection  list  using  the  locations  determined  during  seeding  operations. 

7.3  DETECTION  OF  ALL  TARGETS  OF  INTEREST 

No  dynamic  survey  target  was  selected  within  60  cm  of  six  of  the  seed  items  placed  at  the  former 
WMA.  The  majority  of  the  missed  seeds  (five  of  the  six)  were  in  TO  17,  where  geologic  response 
was  far  more  variable  than  either  of  the  other  two  sites.  The  final  missed  seed  was  in  TO20  Area 
B.  Despite  the  offsets,  the  MetalMapper  operator  was  able  to  correctly  locate  two  of  the  six  with 
enough  accuracy  to  collect  a  MetalMapper  point  within  40  cm  of  the  seed  item.  Table  7-1 
indicates  which  seeds  were  missed  and  comments  on  the  reasons  for  the  misses. 
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Table  7-1:  Seeds  without  Dynamic  Target  within  60  cm 


Seed  ID  / 
Nearest  Target 

Description 

Depth 

(cm) 

Seed  to 
Target 
Offset  (cm) 

Reason  for  Miss 

WSO-06  /  50 

81 -mm  mortar 

60 

74 

Possibly  GPS  datum  offset;  possibly  geology. 
MetalMapper  point  collected  within  40  cm. 

WSO-67  /  885 

37-mm  projectile 

24 

70 

Nearby  geologic  response  greater  than  response  from 
seed. 

WSO-73  /  943 

37-mm  projectile 

16 

82 

Nearby  geologic  response  greater  than  response  from 
seed. 

WSO-93/810 

37-mm  projectile 

15 

65 

Geologic  response  in  area  greater  than  response  from 
seed.  MetalMapper  point  collected  within  40  cm. 

WSO-94  /  787 

60-mm  mortar 

37 

63 

Geologic  response  in  area  greater  than  response  from 
seed. 

WSO-98  /  885 

37-mm  projectile 

15 

895 

Not  within  EM61  survey  area.  Apparently  added  to 
target  list  in  incorrect  location. 

7.4  CORRECTLY  IDENTIFY  SEED  ITEMS  IN  THE  INSTRUMENT  VERIFICATION 
STRIP 

All  IVS  seeds  used  during  the  project  were  37-mm  projectiles  buried  at  a  depth  of  10  cm,  and  a 
total  of  57  measurements  were  taken  over  IVS  items  during  the  project.  Parsons  analyzed  the 
IVS  data  while  on  site,  using  both  in- air  and  ground-based  background  measurements  to  correct 
the  IVS  data  points.  Use  of  the  ground-based  background  correction  data  typically  provided 
better  results  than  the  in-air  points,  although  even  those  results  were  not  ideal.  While  the  IVS 
data  generally  matched  some  item  in  the  library  well  (>  0.9  confidence  metric),  a  correct  match 
to  a  37-mm  projectile  was  inconsistent.  A  total  of  24  of  the  57  measurements  returned  either  a 
37-mm  projectile  or  small  ISO  result  with  a  confidence  metric  above  the  required  0.8.  The 
remaining  results  generally  trended  toward  larger  ordnance  such  as  60-mm  mortars,  seemingly 
indicating  that  background  at  the  seed  locations  was  adding  more  response  than  was  being 
removed  by  the  background  correction.  This  was  confirmed  by  the  use  of  the  in-air  background 
measurements  for  point  correction,  which  removed  less  background  response  from  the  IVS 
points.  Library  matches  for  the  points  corrected  with  the  in-air  data  were  consistently  larger  than 
a  37-mm  projectile,  and  the  smallest  result  returned  for  these  data  was  a  57-mm  projectile.  In 
addition  to  the  mismatches  with  regard  to  seed  size,  4  of  the  57  IVS  results  failed  to  match 
anything  in  the  library  with  a  confidence  metric  greater  than  0.8. 

Given  the  known  issue  of  significant  background  variance  across  the  site,  the  IVS  failures  were 
not  considered  problematic  with  regard  to  the  equipment  in  the  field.  It  was  known  almost  from 
the  beginning  of  the  project  that  the  standard  background  correction  and  UX- Analyze  library 
matching  processes  Parsons  typically  uses  for  classification  were  not  going  to  be  effective  for 
this  site.  Therefore,  the  IVS  data  were  considered  sufficient  to  show  that  the  MetalMapper  was  in 
working  order. 

7.5  PRODUCTION  RATE 

MetalMapper  data  were  collected  over  1,031  discrete  dynamic  target  locations  over  16  days  of 
MetalMapper  use,  which  corresponds  to  a  daily  rate  of  approximately  64  points  per  day. 
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Included  in  the  16  days  were  2  full  days  spent  either  fixing  the  MetalMapper  or  moving  it  from 
one  AOI  to  another  (i.e.,  MetalMapper  use  but  no  data  collection).  If  only  data  collection  time  is 
considered,  the  production  rate  was  closer  to  74  points  per  day,  which  is  still  below  the  goal  of  at 
least  100  points  per  day. 

The  lower-than-expected  production  rates  were  entirely  due  to  site  conditions,  primarily 
topography  and  the  effects  of  geology  on  the  MetalMapper.  Transition  of  the  MetalMapper  from 
point  to  point  was  slowed  by  rocky  topography,  which  took  time  to  navigate  safely.  In  addition, 
changing  geologic  response  made  finding  subsurface  sources  using  the  dancing  arrows  display 
on  the  MetalMapper  screen  extremely  difficult  and  time  consuming.  There  were  also  many  cases 
where  it  became  clear  following  data  analysis  and  the  intrusive  effort  that  numerous  dynamic 
targets  were  selected  due  to  elevated  geologic  response  rather  than  an  actual  subsurface  source, 
so  a  great  deal  of  time  was  spent  searching  for  nonexistent  targets.  While  BTG’s  in-field  QC 
software  was  used  along  with  the  dancing  arrows  and  was  helpful  in  getting  the  MetalMapper 
positioned  over  actual  subsurface  sources,  its  use  also  resulted  in  a  significant  number  of  re¬ 
shots.  A  total  of  1,288  points  were  collected  over  the  1,031  dynamic  targets  on  the  cued  target 
list,  corresponding  to  a  re-shot  rate  of  nearly  25%. 

Parsons  cued  data  collection  production  rates  on  most  other  Military  Munitions  Response 
Program  sites  are  upwards  of  300  points  per  day,  so  the  production  rates  measured  at  the  former 
WMA  are  a  significant  decrease.  However,  nothing  has  been  factored  in  to  the  WMA  production 
rates  that  seems  like  it  could  be  easily  modified  to  vastly  increase  production  rates  on  a  larger- 
scale  project. 

The  Demonstration  Plan  for  the  project  also  indicated  that  pre-processing  time  for  each  target 
should  be  less  than  3  minutes.  Pre-processing  time,  considered  to  be  the  amount  of  time  needed 
to  get  the  .tern  files  collected  by  the  MetalMapper  into  a  format  usable  by  post-processing 
software  such  as  Geosoft  or  MATLAB,  is  a  matter  of  minutes  for  a  day’s  worth  of  data  using 
currently  available  software.  This  metric  can  be  safely  removed  from  consideration  for  future 
projects. 

7.6  CUED  INTERROGATION  OF  ANOMALIES 

An  initial  modeled  location  for  each  MetalMapper  target  was  determined  using  an  in-field  QC 
program,  and  a  re- shot  was  collected  for  any  target  for  which  the  source  modeled  farther  than 
30  cm  from  the  collected  point.  No  additional  re-shots  were  collected  if  a  source  did  not  model 
underneath  the  first  re-shot  location.  The  field  team  did  not  have  sufficient  time  to  re-shoot  any 
targets  that  did  not  have  sources  within  40  cm  of  the  collected  location  despite  the  appearance  of 
a  modeled  source  within  30  cm  of  the  collection  point  in  the  field  (i.e.,  the  final  model  did  not 
agree  with  the  in-field  model). 

7.7  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TARGETS  OF  INTEREST 

Nine  seed  items  placed  within  the  demonstration  areas  were  incorrectly  classified  as  either 
clutter  or  soil  response  on  BTG’s  final  dig  list.  One  of  these,  target  73  (37-mm  projectile)  did  not 
have  a  MetalMapper  point  collected  within  40  cm  of  its  actual  location.  The  nearest 
MetalMapper  point  was  74  cm  offset,  so  the  “soil  response’’  classification  is  likely  correct.  For 
the  remaining  eight  misses,  BTG  concluded  that  the  effects  of  the  background  response  due  to 
site  geology  were  too  significant  to  allow  the  correct  classification  of  these  targets  regardless  of 
processing  method.  Therefore,  the  project  failed  the  performance  objective  of  correctly 
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identifying  all  TOI  at  the  site.  BTG’s  memorandum  describing  the  incorrect  classification  of  the 
seeds  and  their  conclusion  that  all  targets  at  the  site  needed  to  be  dug  to  recover  all  of  the  TOI  is 
included  as  Appendix  B. 

7.8  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TARGETS  OF  INTEREST 

Because  BTG  concluded  that  all  targets  at  the  site  needed  to  be  excavated  to  ensure  the  recovery 
of  all  of  the  TOI  (Appendix  B),  the  project  failed  the  performance  objective  of  reducing  clutter 
digs  by  more  than  75%. 

7.9  CORRECT  SPECIEICATION  OF  NO-DIG  THRESHOLD 

BTG’s  stop-dig  point  in  their  submitted  dig  list  was  at  target  295  of  939  (Appendix  B).  The  eight 
seeds  classified  incorrectly  on  this  list  were  scattered  between  spots  370  and  692  in  the  list.  As  a 
result,  the  stop-dig  point  was  specified  incorrectly.  Because  the  misclassifications  were  caused 
by  geologic  response  drowning  out  the  signal  from  the  seed  items,  BTG  concluded  that  there  was 
no  method  of  reprocessing  the  data  that  would  result  in  the  missed  items  being  moved  up  on  the 
dig  list  to  the  point  that  a  new  stop-dig  threshold  would  be  effective  in  significantly  reducing 
clutter  digs  at  the  site. 

7.10  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE  ANALYZED 

Because  the  no-dig  threshold  was  specified  incorrectly  based  on  geologic  response  masking  the 
response  from  seed  items,  it  is  apparent  that  more  targets  should  have  been  classified  as  “can’t 
analyze’’  due  to  significant  geologic  response.  However,  there  does  not  appear  to  be  a  suitable 
method  for  determining  which  targets  originally  classified  as  due  to  geology  should  be  re¬ 
classified  as  “can’t  analyze’’.  Because  the  correct  number  of  “can’t  analyze’’  targets  is 
indeterminable,  a  specific  percentage  has  not  been  calculated.  Were  all  of  the  targets  currently 
classified  as  geologic  response  converted  to  “can’t  analyze’’,  the  percentage  of  “can’t  analyze’’ 
targets  would  be  significantly  higher  than  the  acceptable  rate  of  5%. 

7.11  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

Because  some  seeds  were  incorrectly  identified  as  non-TOI,  and  in  some  cases  were  classified  as 
geologic  response,  the  target  parameters  (i.e.,  locations)  for  the  sources  were  also  incorrect. 
Because  use  of  the  MetalMapper  for  classification  at  the  former  WMA  did  not  allow  for 
classification  of  all  TOI,  its  use  on  future  projects  is  not  recommended.  Therefore,  offset  data 
were  not  calculated  for  sources  that  were  identified  as  metallic  sources. 
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8.0  COST  ASSESSMENT 


The  cost  assessment  includes  costs  for  project  planning  (demonstration  plan  and  health  and 
safety  plan),  site  preparation  (archaeological  and  biological  surveys,  surface  sweep,  and  blind 
seeding,  EM61  transect  surveys,  EM61  detection  surveys,  military  munitions  cued  data 
collection  and  analysis  costs,  and  intrusive  costs. 

8.1  COST  MODEL 

The  cost  model  for  the  Waikoloa  demonstration  includes  the  total  cost  of  the  project.  The  total 
cost  includes  the  seeding  operation,  MetalMapper  operations,  processing,  and  intrusive 
operation.  Estimates  for  each  operation  are  listed  in  Table  8-1. 
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Table  8-1:  Details  of  Costs  Tracked 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

Project  Planning 

Demonstration  Plan 

Health  and  Safety  Plan 

$25,400 

Site  Preparation 

Archaeological  survey 

Biological  survey 

Surface  sweep 

Blind  seeding 

$45,100 

EM61  Survey 

EM61  transect  surveys 

EM61  detection  surveys  (5  acres) 

$55,700 

Military  Munitions 

Data  Collection 

Cued  data  collection  for  1,288  points  over 
1,032  anomalies 

Per  target 

$76,100 

$74 

Intrusive  Investigations 

All  costs  related  to  the  intrusive 
investigation  (1,880  results  for  1,032 
anomalies  based  on  multiple  items  recovered 
from  anomaly  locations) 

Cost  per  anomaly  to  intrusively  investigate 

$220,000 

$213 
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9.0  IMPLEMENTATION  ISSUES 

There  were  a  few  notable  implementation  issues  regarding  the  Waikoloa  project: 

•  The  largest  implementation  issue  was  a  GPS  datum  discrepancy  at  TO20B.  All  previous 
projects  have  used  the  UTM  NAD  83  datum  (United  States  specific)  for  dynamic  data 
collection  as  opposed  to  the  other  possibility,  the  UTM  WGS84  datum  (world-wide).  The 
MetalMapper  collection  software  assumes  the  WGS84  datum,  which  created  an 
approximately  75-cm  difference  between  the  detection  data  target  locations  shown  on  the 
screen  and  the  actual  source  locations  in  the  field.  The  MetalMapper  collection  team 
generally  started  trying  to  relocate  selected  targets  at  the  anticipated  location  to  the 
southwest  of  the  point  shown  on  the  screen,  but  it  was  not  a  perfect  solution.  To  ensure 
that  all  detected  targets  were  excavated  as  intended.  Parsons  implemented  the  intrusive 
process  detailed  in  Section  5.6,  which  considered  both  the  dynamic  target  location  and 
the  MetalMapper  data  collection  point.  All  dig  list  decisions  were  judged  according  to  the 
source  recovered  at  the  MetalMapper  collection  point. 

•  Data  collection  went  reasonably  smoothly,  but  a  great  deal  of  effort  needed  to  be  placed 
on  identifying  sites  suitable  for  traverse  by  the  forklift  used  to  transport  the  MetalMapper. 
There  were  very  few  locations  within  the  project  boundaries  that  were  flat  enough  and 
that  contained  few  enough  jagged  lava  rocks  for  the  MetalMapper  to  be  used  efficiently. 
Even  if  the  TOI  detection  results  had  been  more  promising,  the  use  of  the  MetalMapper 
at  sites  similar  to  those  chosen  for  the  project  would  have  been  extremely  limited  by 
terrain. 

•  In  addition  to  the  issues  noted  above  with  regard  to  transporting  the  MetalMapper  across 
the  data  collection  areas,  it  was  fairly  difficult  just  getting  the  field  team  out  to  the 
collection  areas  in  TO20.  At  least  a  day  was  lost  during  data  collection  due  to  truck  tires 
flattened  during  the  drive  out  to  the  site.  Replacing  the  standard  tires  on  the  rental  trucks 
used  for  the  project  with  larger,  reinforced  tires  was  successful  in  eliminating  further 
delays  but  was  not  anticipated  at  the  beginning  of  the  project. 
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APPENDIX  A 
Black  Tusk  Geophysics 

Processing  of  Geonics  EM61  and  MetalMapper  Data 
at  the  Former  Waikoloa  Maneuver  Area 


BTG  Processing  of  Geonics  EM61  and 
MetalMapper  Data  at  the  Former 
Waikoloa  Maneuver  Area 

Introduction 

Advanced  EMI  data  were  collected  on  the  Former  Waikoloa  Maneuver  Area  at  three  sites:  Task  Order 
(TO)  17,  T020  Area  A  and  TO20  Area  B  (Figure  1).  Geonics  EM61  data  were  acquired  over  each  site 
(Figures  2-4)  and  anomalies  were  flagged  for  further  investigation  from  the  resulting  data  map.  Cued 
interrogation  data  were  subsequently  acquired  with  the  MetalMapper  at  each  of  the  flagged  anomalies. 

GPS  issues  resulted  in  difficulties  with  the  resulting  survey  data.  At  TO20  Area  A,  the  GPS  base  station 
was  configured  assuming  the  NAD83  datum.  However,  the  MetalMapper  GPS  unit  assumed  the  WGS84 
datum.  This  inconsistency  produced  a  systematic  shift  between  EM61  anomaly  flag  locations  and  the 
locations  at  which  the  MetalMapper  visited.  Partway  through  the  data  collection  effort,  this  shift  was 
recognized  by  the  MetalMapper  data  collection  team.  The  data  acquisition  team  attempted  to 
compensate  for  the  shift  by  repositioning  the  MetalMapper  towards  the  Southwest  at  each  anomaly. 

The  significant  background  soil  response  at  Waikoloa  presented  additional  challenges  for  target  picking 
and  classification.The  magnetic  geology  produces  a  large  amplitude  background  instrument  response 
that  varies  spatially  across  the  site.  The  spatially  varying  geology  and  ground  clearance  variations  due  to 
cart  movement  can  produce  data  anomalies  that  exceed  the  target  picking  threshold.  In  addition  to 
difficulties  with  target  picking,  the  background  soil  response  can  bias  estimated  target  parameters  when 
inverting  data.  Therefore,  special  care  is  required  when  removing  or  accounting  for  the  magnetic  soil 
background  response. 

There  were  two  main  objectives.  The  first  was  to  develop  a  strategy  for  defining  dig  locations  for  the  dig 
or  interrogation  teams.  A  successful  approach  would  (1)  overcome  the  systematic  shift  between  EM61 
anomaly  locations  and  the  locations  visited  by  the  MetalMapper,  and  (2)  avoid  unnecessary  digs  due  to 
spurious  anomaly  picks  due  to  magnetic  geology.  An  approach  was  adopted  that  involved  identifying 
anomalies  where  Geonics  EM61  picks  and  MetalMapper  source  locations  are  far  enough  far  apart  such 
that  both  locations  need  to  be  interrogated  (i.e.  swept  by  the  dig  team  with  a  handheld  Minelabs 
sensor,  and  dug,  if  metal  is  determined  to  be  present).  For  anomalies  where  MetalMapper  location  and 
EM61  flag  are  within  0.5  m,  then  it  is  only  necessary  to  dig  at  the  EM61  flag. 

The  second  objective  was  to  determine  if  it  is  possible  to  accurately  estimate  dipole  source  parameters 
for  targets  buried  in  the  highly  magnetic  geology  of  Waikoloa.  The  standard  survey  approach  is  to 
collect  background  measurements  at  locations  close  to  an  anomaly,  and  then  to  subtract  the 
background  measurement  from  the  anomaly  data.  This  "background  corrected"  data  can  then  be 
inverted  using  the  simplified  model  of  a  target  in  free-space.  Unfortunately,  there  were  an  insufficient 


number  of  background  soundings  to  reliably  background  correct  the  data.  In  addition,  the  spatial 
variability  of  the  background  geology,  strength  of  the  magnetic  field,  and  the  effect  of  inconsistent 
ground  clearance  height  between  anomaly  data  collection  and  background  measurements  can  produce 
inaccurate  estimates  of  the  background  response.  We  adopted  an  approach  where  the  background 
response  was  estimated  as  part  of  a  constrained  inversion  process.  More  specifically,  a  deep  dipole 
source  was  used  to  model  the  effect  of  magnetic  soil. 

This  document  outlines  the  dig  location  strategy  and  the  inversion  approach.  Our  work  was  completed 
as  part  of  ESTCP  MR-201226  and  SERDP  MR-2318. 

Determining  if  a  target  is  a  soil  anomaly 

Electromagnetic  induction  instruments  are  sensitive  to  ferri-magnetic  minerals  such  as  magnetite. 

When  these  minerals  are  exposed  to  a  magnetic  field,  there  is  a  non-instantaneous  change  in 
magnetization.  Pulse  induction  instruments  such  as  the  MetalMapper,  TEMTADS  and  MPV  measure  the 
secondary  field  in  the  off  time  produced  by  the  magnetic  grains  as  the  aligned  magnetization  relaxes  to 
a  more  random  state.  The  secondary  field  amplitude  is  a  function  of  the  concentration  of  ferri-magnetic 
minerals  in  the  soil,  the  distribution  of  time  constants  characterizing  the  relaxation  times,  and  the 
geometry  between  the  transmitters,  receivers,  and  ground.  The  factors  that  determine  the  strength  of 
the  magnetic  soil  response  are  outlined  in  detail  in  the  final  reports  for  SERDP  projects  MM-1414  and 
MM-1573. 

The  Geonics  EM61  was  used  to  map  the  Waikoloa  sites.  The  EM61  has  a  coaxial  transmitter/receiver 
geometry  that  produces  the  maximum  coupling  to  magnetic  soil.  As  a  result,  we  can  use  the  EM61  data 
to  map  the  approximate  variability  in  magnetic  soil  response.  Since  the  EM61  survey  did  not  have  ''in¬ 
air''  measurements  to  establish  an  absolute  zero  level,  a  DC  shift  was  estimated  for  each  data  segment 
such  that  the  shift  between  adjacent  data  segments  were  minimized  when  assembling  the  data  map. 
Figure  5-8  compares  the  along  line,  median  filtered  data  provided  by  Parsons  to  data  re-leveled  to  show 
how  the  background  response  varies  across  the  three  areas.  The  third  time  channel  is  gridded  to  form 
the  image.  In  both  the  median  filtered  data  and  the  data  leveled  to  show  geology,  there  is  a  short 
wavelength,  along-line  oscillation  of  the  data  related  to  the  bouncing  of  the  EM61  cart  producing  a 
periodic  change  in  ground  clearance  height.  The  regions  of  negative  values  in  the  median  filtered  data 
image  are  due  to  filtering  artifacts.  T017  has  the  highest  level  of  the  magnetic  geology  noise  with 
magnetic  soil  responses  exceeding  35  mV  in  the  3^*^  time  channel  (Figure  8).  The  median  filter  was  not 
able  to  remove  the  linear  geologic  features  in  the  data.  Distinct  anomalies  related  to  targets  are  not 
easily  identified  in  the  data,  and  numerous  soil  related  anomalies  picked  for  interrogation  would  be 
expected. 

In  order  to  identify  cases  where  the  MetalMapper  measured  data  over  a  magnetic  geology  anomaly 
instead  of  a  metallic  target,  we  compare  the  MetalMapper  data  with  a  soil  model.  Background 
measurements  were  used  to  determine  the  MetalMapper's  characteristic  magnetic  soil  response. 

Figure  16  plots  the  background  response  for  five  of  the  background  measurements.  Receiver  RxO  was 
not  working  properly  during  these  measurements.  The  measurements  were  corrected  using  in-air 


measurements.  When  the  MetalMapper  is  parallel  to  flat  ground,  we  would  expect  that  the  coaxial 
component  of  the  secondary  response  measured  by  the  center  receiver  (Rx3)  would  be  at  a  maximum. 
Conversely,  the  transverse  components  would  be  null-coupled.  For  example,  the  X  and  Y-component 
receivers  should  measure  only  noise  when  the  Z-component  transmitter  is  firing.  When  the  instrument 
is  not  parallel  to  the  ground  surface,  there  will  be  some  signal  in  these  transmit/receive  pairs. 


The  first  step  to  building  a  soil  model  was  to  use  the  decays  measured  by  the  MetalMapper  to 
determine  how  the  MetalMapper  transmitter  waveform  modified  the  characteristic  1/t  decay  of  the 
magnetic  soil.  The  decay  characteristic  is  independent  of  the  ground  clearance  height  and  orientation  of 
the  sensor  relative  to  the  soil  surface.  The  consistent  decay  character  for  the  different  Tx/Rx 
combinations  can  be  seen  in  Figure  16.  The  decay  character  is  well  modeled  by  the  function: 


1 

t  +  At 


This  function  is  the  characteristic  1/t  VRM  response  convolved  with  a  finite  pulse  transmitter  waveform. 


If  we  assume  that  the  ground  clearance  height  and  sensor  orientation  is  approximately  constant  over 
the  course  of  the  survey,  then  the  relative  strength  of  the  magnetic  soil  response  between  each  Tx/Rx 
combination  can  be  assumed  to  be  constant  during  the  survey.  The  background  measurements 
suggested  that  the  variations  in  the  relative  amplitude  of  soil  responses  between  each  Tx/Rx 
combination  were  not  significant.  Therefore  it  is  reasonable  to  construct  a  soil  model  consisting  of  the 
decay  character  f(t)  which  multiplies  an  amplitude  for  each  Tx/Rx  combination.  This  result  simplifies  the 
soil  fitting  analysis.  Instead  of  requiring  potentially  inaccurate  estimates  of  ground  clearance  height  and 
sensor  orientation  to  model  the  background  response,  we  can  simply  estimate  a  single  scaling 
parameter  that  multiplies  our  soil  model.  If  the  ground  clearance  and  sensor  orientation  is  constant 
during  the  survey,  the  value  of  the  single  scaling  parameter  provides  a  measure  of  the  soil  magnetic 
susceptibility. 


For  each  MetalMapper  measurement,  the  soil  model  is  fit  to  the  measured  data.  The  misfit  between 
the  best  fit  soil  model  and  data  reflects  the  likelihood  that  the  measurement  was  acquired  in  the 
presence  of  a  metallic  target.  Figure  17  has  an  example  of  soil  model  fitting  using  data  over  Target  785 
on  the  Western  edge  of  the  T017  area.  The  absolute  value  of  the  observed  data  is  plotted  as  blue 
circles,  with  a  solid  blue  circle  indicating  positive  data  and  unfilled  (i.e.  white)  circles  with  a  blue  outline 
indicating  negative  data.  A  black  dashed  line  indicates  the  best  fit  soil  model.  There  is  a  very  good 
match  between  the  observed  data  and  the  soil  model.  For  the  null-coupled  Tx/Rx  combinations,  the  soil 
model  predicts  some  signal.  This  is  consistent  with  analysis  of  the  background  data  files  and  suggests 
that  the  MetalMapper  was  consistently  slightly  tilted  when  acquiring  the  cued  data.  When  reviewing 
inversion  results,  the  analyst  is  provided  a  soil  misfit  to  assist  in  determining  if  the  data  anomaly  is  due 
to  soil. 


Selecting  Dig  locations 

Due  to  an  inconsistency  in  the  datum  assumed  for  the  GPS  rover  and  base  station,  there  were 
systematic  errors  in  MetalMapper  positioning  when  data  were  collected  in  TO20  Area  B.  Figure  10  to 
Figure  12  compares  the  locations  at  which  the  MetalMapper  data  were  collected  to  the  locations  of 
targets  picks  from  the  Geonics  EM61  data. 

Figure  10  summarizes  location  errors  for  T017.  Approximately  90%  of  MetalMapper  anomalies  were 
collected  within  50  cm  of  the  EM61  flag  (black  triangle  and  V  symbols).  There  is  an  additional  peak  in 
the  histogram  at  approximately  0.5  m.  This  peak  is  related  to  the  survey  methodology  -  during 
recollects  of  a  sounding,  the  field  crew  would  not  acquire  additional  soundings  at  locations  greater  than 
approximately  0.5  m  from  the  original  anomaly  flag.  The  differences  between  MetalMapper  sounding 
locations  and  EM61  target  picks  were  similar  in  TO20  Area  A  (Figure  11).  Approximately  85%  of 
MetalMapper  anomalies  were  acquired  within  50  cm  of  the  EM61  flag,  and  there  was  again  a  peak  at 
approximately  0.5  m  from  the  original  anomaly  flag.  Figure  12  summarizes  the  differences  for  TO20  Area 
B.  Less  than  62%  of  the  anomalies  had  data  acquired  within  0.5  m  of  the  EM61  pick.  Figure  13  shows 
the  consistent  South-west  bias  between  the  MetalMapper  locations  on  top  of  the  EM61  data  for  TO20 
Area  B. 

A  decision  logic  for  choosing  dig  coordinates  was  established  during  a  meeting  with  members  from 
Parsons,  Black  Tusk  Geophysics,  the  ESTCP  program  office,  and  the  USAGE.  Figure  14  summarizes  the 
decision  logic  for  choosing  target  dig  coordinates.  In  TO20  Area  B,  inaccurate  flag  locations  were  given 
to  the  MetalMapper  data  collection  team.  Therefore,  all  EM61  flag  locations  in  TO20  Area  B  would  be 
investigated,  and  all  MetalMapper  source  locations  that  measured  non-soil  like  data  would  also  be 
investigated.  For  TO20  Area  A  and  T017,  it  is  not  necessary  to  investigate  every  EM61  flag.  For  cases 
where  the  MetalMapper  location  and  EM61  flag  are  greater  than  0.5  m  apart,  and  analysis  of  the 
MetalMapper  data  suggests  a  non-soil  target,  the  dig  team  will  only  have  to  investigate  at  the  location 
of  the  MetalMapper  source  and  not  at  the  EM61  flag. 


Inversion  of  MetalMapper  data  at  Waikoloa 

The  standard  procedure  when  collecting  cued  EMI  data  is  to  take  background  measurements.  These 
background  measurements  are  then  subtracted  from  data  acquired  over  targets.  These  ''background 
corrected"  data  can  be  modeled  as  a  dipole  in  free-space.  This  approach  requires  a  background 
measurement  acquired  close  enough  to  each  anomaly  such  that  there  is  an  accurate  estimate  of  the 
background  signal  for  each  anomaly.  At  sites  with  large  magnetic  soil  response,  this  approach  can  have 
limited  success.  This  is  because  small  changes  in  ground  clearance  and  changes  in  orientation  of  the 
MetalMapper  can  result  in  a  significant  change  in  amplitude  of  the  soil  response.  A  change  in  ground 
clearance  of  only  a  few  centimeters  can  result  in  a  change  of  background  response  of  several  tens  of 
percent  (SERDP  MR-1573  Final  Report).  Overestimating  or  underestimating  the  background  response 
will  reduce  the  accuracy  with  which  the  dipole  polarizabilities  are  estimated  as  shown  in  Figure  15  for 
TEMTADS  2x2  data  collected  at  Waikoloa. 


An  alternative  to  subtracting  a  background  measurement  prior  to  the  inverting  the  data  is  to  solve  for 
the  soil  response  as  part  of  the  data  inversion.  Several  modeling  based  approaches  were  considered  in 
SERDP  1573.  It  was  noted  in  SERDP  1573  that,  for  cued  MetalMapper  data,  the  magnetic  soil  response 
could  be  modeled  well  with  a  deep  dipole  source.  For  the  Waikoloa  cued  MetalMapper  data,  we 
developed  a  multi-dipole  source  inversion,  where  the  location  of  a  deep  source  is  fixed  beneath  the 
center  of  the  array.  This  single  deep  source  will  model  the  background  response  due  to  soil,  and  the 
remaining  sources  will  model  any  buried  metallic  targets. 

Figure  17  contains  a  typical  result  when  fitting  cued  MetalMapper  data  taken  over  a  soil  anomaly  with  a 
dipole  source.  Three  lines  are  shown  on  each  plot:  observed  data  are  plotted  in  blue,  the  best  fit  soil 
model  is  plotted  in  black,  and  the  predicted  data  when  using  a  single  source  inversion  is  plotted  in 
green.  The  observed  data  is  fit  well  with  both  the  best  fit  soil  model  and  the  single  deep  source.  The 
recovered  dipole  polarizabilities  and  relative  depths  when  inverting  for  1,  2,  and  3  sources  are  shown  in 
Figure  18.  The  single  source  inversion  shows  that  the  soil  can  be  fit  with  a  plate-like  target  oriented  with 
the  normal  of  the  plate  parallel  to  the  horizontal  direction.  Solving  for  two  dipole  sources  results  in  a 
deep  plate-like  target,  and  a  shallow  small  amplitude  target  that  fits  noise.  Three  sources  adds  a  second 
small  amplitude  target  at  the  surface.  To  represent  the  soil  response,  we  choose  to  use  the  simpler 
model  of  a  single  deep  source.  Analysis  of  background  measurements  corrected  with  an  in-air  sounding 
were  used  to  determine  that  a  source  depth  of  95  cm  would  model  the  soil  response  well.  Figure  19 
shows  that  there  is  a  second  local  minimum  in  the  fitting  problem  where  the  soil  is  fit  well  with  two 
sources:  one  source  at  40  cm  and  a  second  source  at  approximately  95  cm. 

IVS  data  were  used  to  test  the  deep-source  soil  modeling  methodology.  The  IVS  contains  3  targets. 
Targets  1  and  3  are  the  same  37  mm.  Targets  1  and  2  are  horizontal  and  buried  such  that  the  geometric 
center  of  the  target  is  at  a  depth  of  10  cm.  Target  3  is  vertical,  with  the  geometric  center  of  the  target 
being  at  12  cm.  The  IVS  data  were  inverted  using  in-air  measurements  for  background  correction,  and 
the  data  were  fit  using  (a)  single  source  inversions,  (b)  two  source  inversions  in  which  the  location  of 
both  sources  were  unconstrained,  and  (c)  two  source  inversions  in  which  the  location  of  one  of  the 
sources  was  constrained  to  be  deep  and  centered  on  the  MetalMapper  instrument.  Figure  20  compares 
the  recovered  polarizabilities  from  the  three  types  of  inversions.  When  inverting  for  a  single  source,  the 
amplitudes  of  the  polarizabilities  are  overestimated  since  the  source  is  trying  to  fit  both  the  37  mm 
projectile  and  the  soil  response  simultaneously.  Inverting  for  a  pair  of  sources  improves  the  amplitude 
of  the  polarizability  estimates.  Inverting  for  a  pair  of  sources,  with  one  source  fixed  deep  to  represent 
the  background  soil  response  produces  similar  results  to  the  unconstrained  two  source  response  for  IVS 
target  1  and  IVS  target  2,  with  only  marginal  improvements  in  the  spread  of  recovered  polarizabilities. 
There  is  a  more  pronounced  improvement  when  fitting  IVS  Target  3,  with  a  larger  spread  in  recovered 
polarizabilities  when  inverting  for  two  unconstrained  sources.  Fitting  IVS  target  3  with  a  deep  source 
improves  the  accuracy  of  the  recovered  polarizabilities. 

Figure  21  compares  the  size  vs.  decay  feature  plot  for  the  IVS  inversions.  Figure  21(a)  shows  that,  when 
inverting  the  data  with  only  a  single  source  inversion,  there  is  a  bias  in  the  recovered  feature  clusters 
since  the  polarizability  size  is  overestimated  and  the  estimated  decay  rate  slower  than  expected  due  to 
the  dipole  source  trying  to  fit  both  the  target  response  and  soil.  When  inverting  the  data  with  two 


unconstrained  sources,  the  second  source  models  the  response  of  the  ground,  and  the  bias  seen  in  (a)  is 
removed  (Figure  21(b)).  The  size  and  decay  features  associated  with  the  source  that  models  the  soil  is 
outlined  with  a  black  dashed  line.  Figure  21(c)  shows  that  the  feature  clusters  are  better  constrained  for 
the  IVS  targets  when  the  second  source  is  constrained  to  be  deep  and  centered  at  the  center  of  the 
anomaly.  In  addition,  using  the  constrained  inversion  results  in  the  size  and  decay  characteristics  of  the 
soil  sources  being  constrained  better  than  in  (b). 

We  processed  the  Waikoloa  MetalMapper  cued  data  using  both  the  soil  modeling  method  described 
above,  and  also  using  the  standard  background  correction  method  using  available  background 
measurements.  One,  two,  and  three  source  inversions  for  the  two  background  correction  methods 
resulted  in  12  models  for  each  anomaly.  From  these  set  of  models,  the  coordinates  of  any  models 
characteristic  of  a  compact  metallic  target  was  recorded  as  a  potential  location  for  a  dig  team  to  visit. 


Deliverables 

PDF  files  were  generated  that  summarized  analysis  for  each  MetalMapper  measurement.  Each  page  of 
the  PDF  files  contains  the  following  information  for  an  anomaly: 

•  The  MetalMapper  CSV  file  name,  MetalMapper  target  location,  EM61  flag  location,  soil  model 
misfit,  distance  between  EM61  flag  location  and  MetalMapper  source  location 

•  The  dig  logic  is  summarized  in  magenta.  The  soil  model  misfit  is  reported. 

•  A  small  chip  of  EM61  data  (as  filtered  by  Parsons)  is  plotted  with  the  MetalMapper  source 
location  and  EM61  flag  location  plotted  on  top.  The  first  time  channel  of  EM61  data  is  plotted 

•  The  location  of  the  anomaly  is  plotted  on  a  map  of  the  EM61  map  of  the  entire  grid. 

•  The  data  fit  is  shown  for  each  transmitter/receiver  combination.  The  best  fit  soil  model  is 
shown  with  black  dashed  lines. 

•  For  anomalies  that  were  not  classified  as  soil  models,  the  recovered  polarizabilities  are  plotted 
in  the  upper  right  of  the  page. 

Figure  22  to  Figure  24  include  examples  of  the  PDF  summaries  for  three  anomalies  in  TO20  Area  B.  The 
information  contained  in  the  PDFs  was  also  provided  in  a  number  of  Excel  spreadsheets.  An  example  of 
the  Excel  spreadsheets  can  be  found  in  Figure  25. 

In  addition  to  the  MetalMapper  analysis,  additional  dig  locations  based  on  MPV  data  were  provided.  A 
summary  of  that  process  is  provided  in  the  report  for  ESTCP  MR-201158. 


Figure  1:  Site  Area.  Three  survey  areas  were  analyzed:  T017,  TO20  A  and  TO20  B. 


Figure  2:  Area  T017.  Geonics  EM61  data  acquired  and  median  filtered  by  Parsons  are  overlaid  on  the  grid  area. 
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Figure  3:  Area  TO20A.  Geonics  EM61  data  acquired  and  median  filtered  by  Parsons  are  overlaid  on  the  grid  area. 


Figure  4:  Area  TO20B.  Geonics  EM61  data  acquired  and  median  filtered  by  Parsons  are  overlaid  on  the  aerial  map. 


Figure  5:  TO20  Area  B.  Data  levelled  to  Show  Geology. 
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Channel  3  -  Median  filtered 


Channel  3  -  levelled  to  show  geology 


6:  TO20  Area  A,  North.  Data  levelled  to  Show  geology. 
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Figure  7:  TO20  Area  A  South. 
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Figure  8:  T017.  Data  leveled  to  show  geology. 


TO20  Area  A  background  locations 


TO20Area  B  background  locations 
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Figure  9:  Location  of  background  soundings.  Magenta  circles  indicate  locations  at  which  background  soundings  were 
collected. 
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Figure  10:  T017.  Comparison  of  EM61  pick  location  and  MetalMapper  sounding  location 


-  count 


Figure  11.TO20  Area  A.Comparison  of  EM61  pick  location  and  MetalMapper  sounding  location 


distance  to  closest  EM61  (m) 


Figure  12:  TO20  Area  B.  Comparison  of  EM61  pick  locations  and  MetalMapper  Sounding  locations. 


Figure  13:  Close-up  of  east  section  of  TO20-Area  B.  In  this  area  there  is  a  consistent  Southwest  bias  between  the  EM61  data 
anomalies,  and  where  the  MetalMapper  acquired  cued  data. 


(a)  TO20  Area  A  and  T017  (b)  TO20  Area  B 

Figure  14:  Decision  logic  for  choosing  target  dig  coordinates.  In  TO20  Area  B,  inaccurate  flag  locations  were  given  to  the 
MetalMapper  data  collection  team.  The  approach  outlined  above  represents  a  conservative  approach:  all  EM61  flag 
locations  would  be  investigated,  and  all  MetalMapper  source  locations  that  measured  non-soil  like  data  would  also  be  dug. 
For  TO20  Area  A  and  T017,  it  is  not  necessary  to  dig  every  EM61  flag. 


(a)  IS02  depth  =  15  cm 


Site1,  TCI,  afiles 
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(a)  IS02  depth  =  30  cm 


Figure  15:  Comparison  of  recovered  polarizabilities  when  inverting  cued  TEMTADS  2x2  data  with  inaccurate  estimation  of 
the  background  response.  For  this  example,  we  use  background  soil  responses  measured  with  a  TEMTADS  2x2  sensor  on 
Waikoloa.  Each  panel  overlays  results  from  N  simulations  falling  within  the  percent  difference  from  the  true  polarizability 
range  in  the  panel  title.  When  the  percent  difference  is  less  than  0,  we  are  overestimating  the  background  response.  As  a 
result,  the  polarizability  amplitude  is  reduced,  and  there  are  poor  estimates  of  the  secondary  polarizabilities.  When  the 
percent  difference  is  greater  than  zero,  the  background  response  is  underestimated,  therefore  leaving  soil  noise  in  the  data 
to  be  inverted.  For  this  case,  the  estimated  polarizability  is  greater  than  the  true  polarizability,  and  the  secondary 
polarizabilities  follow  the  decay  of  the  background  soil.  When  the  IS02  is  at  a  depth  of  30  cm,  these  effects  are  more 
significant,  and  the  ability  to  recover  informative  polarizabilities  is  more  sensitive  to  errors  in  the  background  estimation. 
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Figure  16:  Example  of  background  measurements  corrected  with  air.  Blue  lines  are  for  positive  responses,  and  red  lines 
represent  negative  responses.  Receiver  RxO  was  not  working  properly  during  these  measurements. 


APPENDIX  B 

Failure  Analysis  Memorandum 


Failure  Analysis  Memo 


Site:  Waikoloa 

Analyst:  Black  Tusk  Geophysics 
Data:  MM  Cued 
Date:  January  23,  2015 

Our  stage  1  dig  list  missed  nine  QC  seeds  (see  table  below). 


Table  1:  List  of  nine  missed  QC  seeds.  Last  column  is  a  classification  based  on  data  match  to  a  soil 
model  and  visual  inspection  of  observed  data  and  predicted  polarizabilities. 


Dig 

number 

Anomaly 

Survey 

Area 

Identification 

Depth  (cm) 

Soil 

anomaly? 

370 

402 

TO20-A 

37mm  (TP) 

29 

No 

383 

1026 

T017 

GOrnnn  Mortar 

34 

Yes 

396 

715 

TO20-A 

37nnnn 

25 

No 

427 

203 

TO20-B 

60nnm 

40 

Yes 

528 

391 

TO20-B 

GOrnm  mortar 

45 

Yes 

555 

393 

TO20-B 

60mm  mortar 

45 

Yes 

568 

707 

TO20-A 

37mm 

20 

Yes 

660 

474 

TO20-A 

60mm  mortar 

40 

Yes 

692 

73 

TO20-B 

37mm  TP 

11 

Yes 

The  missed  QC  seeds  were  distributed  amongst  all  three  areas.  The  stop  dig  point  on  our  stage  1 
dig  list  was  dig  number  295.  The  missed  QC  seeds  occurred  between  digs  number  370  to  692. 
(Total  number  of  anomalies  is  939.) 


a.  Analysis  of  the  factors  that  resulted  in  the  misclassification  of  each  missed  seed 

For  each  anomaly  of  the  Waikoloa  MetalMapper  Cued  dataset  we  performed  six  inversions, 
comprising  two  sets  of  three  inversions.  In  both  sets  we  solved  for  (1)  a  single  object  (single 
object  inversion:  SOI);  (2)  two  objects  (201);  and  (3)  three  objects  (301),  resulting  in  six  models 
per  anomaly.  In  the  first  set  of  three  inversions  we  used  data  that  were  background  corrected 
using  the  nearest  in-air  background  measurement.  For  the  two-  and  three-object  inversions,  the 
location  of  one  source  was  fixed  at  the  center  of  the  MetalMapper  array  at  a  depth  of  90  cm. 
Previous  investigations  showed  that  the  fixed  source  can  be  effective  at  representing  the  soil 
component  of  the  signal.  The  single-object  inversion  in  this  set  will  typically  be  unreliable 
because  not  enough  background  will  have  been  removed. 

In  the  second  set  of  three  inversions  we  used  data  that  were  corrected  using  the  nearest  normal 
(close  to  the  ground)  background  measurement.  No  source  locations  were  fixed.  With  a 
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background  measurement  representative  of  the  background  response  at  the  anomaly  location,  this 
second  approach  can  work  well.  Unfortunately,  for  the  Waikoloa  dataset,  the  spatial  variation  in 
background  response  is  large  and  the  spatial  coverage  of  the  background  measurements  is  poor 
(particularly  for  areas  TO  17  and  TO20A).  As  a  result,  frequently  either  too  much  or  not  enough 
background  is  subtracted,  leading  to  poor,  unreliable  inversion  results. 

Our  belief  was  that  this  two-pronged  approach,  which  results  in  a  total  of  twelve  models  per 
anomaly,  would  provide  the  best  chance  of  obtaining  at  least  one  good,  representative  model  per 
anomaly. 

Our  stage  1  dig  list  comprised  two  parts.  For  part  1  (digs  1 — 214)  classification  was  based  on 
polarizability  matching  against  a  reference  library  using  all  three  polarizabilities  and  the  first  40 
time  channels  (up  to  6.41  ms).  For  part  2  (digs  215 — 939),  classification  was  based  on  the  data 
misfit  with  a  soil  model  that  we  developed  from  Waikoloa  MetalMapper  soundings  that  look  like 
pure  soil  response.  For  the  latter,  anomalies  with  data  closely  resembling  the  soil  model  appear 
later  on  the  dig  list.  The  soil  model  misfit  is  calculated  using  an  arbitrary  scale  factor  for  each 
anomaly  to  account  for  amplitude  variations  associated  with  the  varying  strength  of  the  magnetic 
soil  response  across  the  site.  The  transition  point  between  the  two  parts  of  the  dig  list  was  chosen 
by  visual  inspection  of  the  polarizabilities  in  dig  list  order,  with  the  transition  point  positioned 
just  ahead  of  the  point  where  polarizabilities  sorted  by  matching  against  the  reference  library 
returned  a  majority  of  soil  anomalies.  Anomalies  near  the  front  of  the  second  part  of  the  dig  list 
should  represent  anomalies  with  a  substantial  presence  of  metal  in  the  ground,  though  their 
match  to  items  in  the  reference  library  will  not  necessarily  be  good.  This  was  considered  to  be  a 
conservative  approach  for  ensuring  non-soil  anomalies  occur  ahead  of  soil  anomalies. 

Of  the  nine  QC  seeds  we  missed,  only  two  (WK-402  and  WK-715)  were  missed  because  their 
polarizability  misfit  was  too  large  relative  to  the  decision  point  we  chose.  For  the  other  seven 
missed  QC  seeds,  the  data  and  polarizabilities  are  consistent  with  what  we  consider  to  be  a 
typical  soil  response.  For  one  of  these  seven  (WK-73;  37mm  TP  at  1 1  cm  depth;  Figure  1),  the 
offset  between  ground  truth  location  and  MetalMapper  acquisition  location  is  large  (74  cm).  This 
likely  explains  the  soil  response  at  this  anomaly. 

15 

10 
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-5 

-10 

Figure  1.  EM-61  data  in  the  vicinity  of  WK-73  (37mm  TP  at  1 1  cm  depth).  Thin  lines  show 
outline  of  the  MetalMapper.  Broken  black  lines  are  horizontal  inversion  boundary.  Circles  are 
predicted  model  locations.  Black  “X”  is  ground  truth  location  of  the  seed.  Offset  to  center  of  the 
MetalMapper  is  about  74  cm. 
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b.  Description  of  how  the  analysis  procedures  have  been  modified  based  on  the  additional 
information  provided 

QC  seeds  are  typically  missed  because  either  (1)  the  polarizability  match  to  reference  items  is 
somewhat  worse  than  expected  based  on  an  assessment  of  overall  data  quality;  or  (2)  the  seed  is 
a  “one-off’  item  not  discovered  during  training,  and  therefore  not  represented  in  the 
polarizability  reference  item.  The  corrective  measure  to  take  in  both  cases  is  obvious:  either  (1) 
modify  the  decision  point  so  that  items  with  larger  polarizability  misfits  appear  before  the  stop 
dig  point;  or  (2)  add  the  new  item  to  the  polarizability  reference  library.  For  this  dataset, 
however,  the  key  issue  with  most  of  the  missed  QC  seeds  is  that  the  data  contain  no  obvious 
signal  beyond  a  soil  response,  resulting  in  recovered  polarizabilities  that  bear  no  resemblance  to 
those  of  the  associated  library  item(s).  This  is  exemplified  by  anomaly  WK-707  (37mm  ATP  at 
20  cm  depth; 

Figure  2  and  Figure  3).  The  similarity  between  the  observed  data  and  the  soil  model  resulted  in 
this  anomaly  appearing  at  dig  number  568.  There  was  no  possibility  that  this  anomaly  would  be 
dug  early.  A  similar  situation  exists  with  WK-203,  391,  393,  474  and  1026  (See  Appendix  for 
data  and  polarizabilities  of  these  anomalies).  Figure  4  shows  polarizabilities  for  digs  541-576  and 
illustrates  the  difficulty  in  recognizing  two  of  the  missed  QC  seeds  (WK-393  and  WK-707) 
whose  polarizabilities  look  very  much  like  those  from  other  soil  anomalies.  For  this  site,  with 
several  QC  seeds  that  cannot  be  detected  due  to  a  very  strong  soil  response,  and  given  the 
assumption  that  the  objective  at  this  site  is  to  find  all  TOI,  and  assuming  that  the  ground  truth 
information  is  reliable,  the  prudent  approach  is  to  dig  all  anomalies. 
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Figure  2.  Data  for  missed  QC  seed  WK-707  (37mm  APT  at  20  cm  depth).  Blue:  observed  data. 
Green:  predicted  data.  Broken  black  line:  soil  model.  In  general  all  soundings  are  an  excellent 
match  to  the  soil  model  (so  that  the  observed  and  predicted  data  are  obscured  by  the  black  line). 
Receiver  1  (top  row)  data  were  generally  bad  for  all  anomalies  and  were  not  included  in  the 
inversions. 
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T1026-C1009-SOI/1-F 


T1026-C1009-20I/1-P 


T1026-C1009-20I/2-F 


T1026-C1009-30I/1-P 


Figure  3.  Polarizabilities  for  missed  QC  seed  WK-707  (37mm  APT  at  20  cm  depth).  The  first 
six  models  (light  blue  background)  used  data  background  corrected  using  the  nearest  in-air 
measurement  and,  for  the  two-object  (201)  and  three-object  (301)  inversions,  fixing  one  of  the 
sources  at  90  cm  depth.  The  remaining  six  models  used  data  that  were  corrected  using  the  nearest 
near-ground  background  correction.  Red,  black  and  magenta  lines  are  predicted  polarizabilities. 
Broken  grey  lines  are  best  fitting  reference  library  polarizabilities.  The  predicted  polarizabilities 
for  all  models  are  typical  of  a  soil  response.  None  of  the  models  remotely  resemble  any  of  the 
37mm  reference  items. 
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Figure  4.  Polarizabilities  in  dig  list  order  for  digs  number  541  through  576.  Missed  QC  seeds  are 
indicated  by  colored  labels:  WK-393  (60mm  mortar;  dig  555)  and  WK-707  (37mm;  dig  568).  All 
of  the  anomalies  shown  here  have  data  which  closely  match  our  soil  model. 


c.  Evidence  that  the  modified  analysis  scheme  correctly  classifies  the  missed  seeds  and  can 
reasonably  be  expected  to  correctly  classify  all  remaining  TOI. 

Digging  all  anomalies  will  correctly  classify  the  missed  seeds  and  will  correctly  classify  all 
remaining  TOI. 
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Appendix:  Data  and  polarizabilities  for  other  soil-like  missed  QC  seeds 

RxX-TxX  RxY-TxX  RxZ-TxX  RxX-TxY  RxY-TxY  RxZ-TxY  RxX-TxZ  RxY-TxZ  RxZ-TxZ 


Data  for  missed  QC  seed  WK-73:  37mm  TP  at  1 1  cm  depth. 


T73-C82-20I/2-F  T73-C82-30I/1-F  T73-C82-30I/2-F  T73-C82-30I/3-F 


Polarizabilities  for  missed  QC  seed  WK-73:  37mm  TP  at  1 1  cm  depth. 
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Data  for  missed  QC  seed  WK-203:  60mm  at  40  em  depth. 


Polarizabilities  for  missed  QC  seed  WK-203:  60mm  at  40  em  depth. 
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Data  for  missed  QC  seed  WK-391 :  60mm  at  45  cm  depth. 


Polarizabilities  for  missed  QC  seed  WK-391 :  60mm  at  45  cm  depth. 
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Rx7  Rx6  Rx5  Rx4  Rx3  Rx2  Rx1 


RxX-TxX  RxY-TxX  RxZ-TxX 


RxX-TxY  RxY-TxY  RxZ-TxY 


RxX-TxZ  RxY-TxZ  RxZ-TxZ 


Data  for  missed  QC  seed  WK-393:  60mm  at  45  cm  depth. 


Polarizabilities  for  missed  QC  seed  WK-393:  60mm  at  45  cm  depth. 
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RxX-TxX  RxY-TxX  RxZ-TxX 


RxX-TxY  RxY-TxY  RxZ-TxY 


RxX-TxZ  RxY-TxZ  RxZ-TxZ 


Data  for  missed  QC  seed  WK-474:  60mm  at  40  em  depth. 


T474-C536-SOI/1-F 


T474-C536-20I/1  -F  T474-C536-20I/2-F 


T474-C536-30I/1-P 


Polarizabilities  for  missed  QC  seed  WK-474:  60mm  at  40  em  depth. 
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RxX-TxX  RxY-TxX  RxZ-TxX  RxX-TxY  RxY-TxY  RxZ-TxY  RxX-TxZ  RxY-TxZ  RxZ-TxZ 


Data  for  missed  QC  seed  WK-1026:  60mm  at  34  em  depth. 


T1026-C1009-SOI/1-F 


T1026-C1 009-201/1 -P 


T1026-C1009-20I/2-F 


T1026-C1 009-301/1 -P 


Polarizabilities  for  missed  QC  seed  WK-1026:  60mm  at  34  em  depth. 
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